-- Logs begin at Sun 2022-06-26 02:17:15 UTC, end at Sun 2022-06-26 03:32:19 UTC. -- Jun 26 02:50:50.280105 dv-cinder-846842-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 26 02:50:50.288599 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 26 02:50:50.290655 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Jun 26 02:50:50 2022] *** Jun 26 02:50:50.290655 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 26 02:50:50.290655 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 26 02:50:50.290655 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: nodename: dv-cinder-846842-4-iscsi Jun 26 02:50:50.290655 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: machine: x86_64 Jun 26 02:50:50.290655 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: clock source: unix Jun 26 02:50:50.290655 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: pcre jit disabled Jun 26 02:50:50.290655 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: detected number of CPU cores: 4 Jun 26 02:50:50.290655 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: current working directory: / Jun 26 02:50:50.290655 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: detected binary path: /usr/bin/uwsgi-core Jun 26 02:50:50.291171 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: your processes number limit is 47680 Jun 26 02:50:50.291171 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: your memory page size is 4096 bytes Jun 26 02:50:50.291171 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: detected max file descriptor number: 2048 Jun 26 02:50:50.291171 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: lock engine: pthread robust mutexes Jun 26 02:50:50.291171 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: thunder lock: enabled Jun 26 02:50:50.291171 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 26 02:50:50.291171 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 26 02:50:50.333948 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: Python main interpreter initialized at 0x55fe39e9d9c0 Jun 26 02:50:50.333948 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: python threads support enabled Jun 26 02:50:50.333948 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: your server socket listen backlog is limited to 100 connections Jun 26 02:50:50.333948 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: your mercy for graceful operations on workers is 90 seconds Jun 26 02:50:50.334296 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: mapped 403077 bytes (393 KB) for 2 cores Jun 26 02:50:50.334393 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: *** Operational MODE: preforking *** Jun 26 02:50:50.334664 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: *** uWSGI is running in multiple interpreter mode *** Jun 26 02:50:50.334664 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: spawned uWSGI master process (pid: 80092) Jun 26 02:50:50.334847 dv-cinder-846842-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 26 02:50:50.337438 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: spawned uWSGI worker 1 (pid: 80102, cores: 1) Jun 26 02:50:50.337438 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: spawned uWSGI worker 2 (pid: 80103, cores: 1) Jun 26 02:50:50.337438 dv-cinder-846842-4-iscsi devstack@keystone.service[80092]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 26 02:50:55.742629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 26 02:50:55.742629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: from cryptography.utils import int_from_bytes Jun 26 02:50:55.743406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 26 02:50:55.743406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: from cryptography.utils import int_from_bytes Jun 26 02:50:55.747263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 26 02:50:55.747263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: from cryptography.utils import int_from_bytes Jun 26 02:50:55.747703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 26 02:50:55.747703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: from cryptography.utils import int_from_bytes Jun 26 02:50:55.791402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 26 02:50:55.791596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 26 02:50:55.791790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] command line args: [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 26 02:50:55.792057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 26 02:50:55.792234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ================================================================================ {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 26 02:50:55.792485 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] admin_token = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.792727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] config_dir = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.792915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.793097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] config_source = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.793276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] debug = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.793486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.794389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.794498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 26 02:50:55.794602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.794702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 26 02:50:55.794797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] command line args: [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 26 02:50:55.794894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.795029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 26 02:50:55.795138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.795233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ================================================================================ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 26 02:50:55.795326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.795423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_config_append = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.795535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] admin_token = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.795631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.795723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config_dir = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.795821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_dir = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.795920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.796014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_file = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.796124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] config_source = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.796219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.796311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] debug = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.796394 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.796509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.796617 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.796694 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.796808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.796880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_config_append = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797802 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.797980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_dir = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.798087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.798199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.798323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.798432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.798575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.798681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.798786 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.798923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.799036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.799176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.799285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.799390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.799526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.799705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.799827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.799974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] publish_errors = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.800087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.800204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.800345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.800451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.800557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.800660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.800765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.800868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.800976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.801051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.801120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.801227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] standard_threads = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.801299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.801366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.801565 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.801648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.801723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.801888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] use_journal = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.802036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.802145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] use_json = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.802222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] publish_errors = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.802333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] use_stderr = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.802405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] use_syslog = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.802541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.802614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.802682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.802841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.802916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.803084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.803160 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.803317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.803397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.803552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] standard_threads = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.803658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.803783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.803859 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.804016 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.804096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.804191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.804270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.804370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_journal = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.804479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.804563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_json = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.804708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_stderr = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.804787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.804884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] use_syslog = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.804968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.805113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 26 02:50:55.805193 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.805366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.805441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.805605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.805686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.805851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.805926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.806112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.806200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.806355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.806439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.806590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.806665 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.806836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.806915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.807072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.807159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.807313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.807388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.807545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.807620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.807790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.807937 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.808009 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.808076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.808262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.808337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.808499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.808581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.808727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.808807 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.808955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.809036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.809204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.809308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.809438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.809519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.809681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.809757 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.809925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.810007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.810192 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.810266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.810447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.810521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.810688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.810776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.810932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.811008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.811168 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.811250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.811408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.811551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.811631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.811699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.811861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.811948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.812085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.812160 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.812309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.812383 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.812537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.812611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.812765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.812841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.812994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.813075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.813226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.813301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.813470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.813550 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.813697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.813772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.813930 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.814011 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.814179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.814263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.814415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.814496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.814650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.814725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.814882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.814957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.815112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.815191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.815344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.815418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.815575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.815650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.815808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.815883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.816040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.816105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.816273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.816340 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.816500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.816565 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.816810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.816952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.817099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.817207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.817336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.817445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.817445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.817650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.817650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.817885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.817996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.818151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.818257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.818401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.818401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.818622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.818755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.818873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.818873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.819078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.819210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.819323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.819429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.819563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.819669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.819774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.819877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.820019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.820125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.820263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.820263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.820470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.820576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.820706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.820706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.820919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.820919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.821120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.821253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.821364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.821451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.821561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.821634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.821708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.821815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.821895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.821966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.822035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.822151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.822226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.822384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.822475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.822622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.822704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.822858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.822941 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.823094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.823179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.823331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.823413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.823565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.823647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.823798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.823882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.824032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.824107 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.824257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.824340 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.824485 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.824636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.824715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.824790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.824954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.825031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.825183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.825269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.825421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.825509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.825653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.825729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.825893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.825968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.826147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.826239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.826384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.826459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.826620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.826704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.826851 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.826970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.827079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.external = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.827154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.827307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.827380 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.827541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.827628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.827772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.827847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.828009 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.828084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.password = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.828242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.828366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] auth.token = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.828475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.828556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.828706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.828781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.829146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.829231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.829303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.829379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.829450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.829520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.829622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.829711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.external = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.829851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.829929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.830100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.830181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.830344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.830420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.830587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.830666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.password = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.830829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.830905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] auth.token = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.831063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.831139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.831296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.831373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.831523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.831598 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.831759 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.831834 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.832006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.832085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.832241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.832317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.832471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.832546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.832705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.832782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.832948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.833024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.833196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.833273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.833442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.833525 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.833678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.833761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.833919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.833995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.834174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.834261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.834424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.834507 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.834668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.834746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.834898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.834975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.835142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.835218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.835377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.835460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.835625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.835703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.835858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.835934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.836108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.836191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.836344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.836422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.836582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.836658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.836813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.836888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.837055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.837131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.837295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.837370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.837524 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.837598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.837754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.837830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.837993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.838081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.838243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.838319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.838466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.838540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.838713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.838790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.838960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.839044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.839190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.839266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.839417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.839494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.839646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.839729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.839887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.839963 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.840119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.840194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.840871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.841022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.841124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.841200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.841356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.841432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.841594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.841679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.841841 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.841917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.842093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.842175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.842322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.842411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.842548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.842635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.842772 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.842849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.843003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.843091 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.843239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.843316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.843473 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.843549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.843716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.843800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.843944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.844020 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.844184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.844260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.844423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.844511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.844654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.844731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.844887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.844962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.845123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.845210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.845348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.845424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.845580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.845655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.845804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.845887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.846035 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.846127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.846284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.846360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.846518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.846607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.846750 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.846828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.846982 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.847059 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.847223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.847303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.847465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.847538 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850524 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.850990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851055 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851291 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851291 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851401 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851401 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.851746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852383 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852440 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.852965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.853962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.854978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.855134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.855219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.855381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.855458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.855607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.855682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.855845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.855929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.856641 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.856732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.856881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.856958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.857117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.857202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.857357 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.857433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.857591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.857666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.857817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.857901 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.858120 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.858204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.858311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.858388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.858555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.858648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.858785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.858861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.859017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.859092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.859251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.859335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.859485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.859562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.859714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.859789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.859955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.860038 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.860188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.860264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.860418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.860494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.860654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.860738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] role.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.860892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.860968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] role.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.861117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.861192 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.861363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.861518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.861592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.861661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.861838 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.861914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.862096 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.862187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.862343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.862419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.862634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.862718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.862817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.862901 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.863055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.863132 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.863280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.863355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.863523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.863607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.863753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.863830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.863993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.864069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.864235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.864313 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.864478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.864632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.864724 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.864866 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.864971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.865115 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.865215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.865358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.865470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.865603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.865718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.865847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.865949 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.866097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.866219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.866350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.866465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.866598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.866710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.866833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.866956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.868987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.868987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.869955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870261 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.870975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871045 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871401 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.871992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.872061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.872141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.872213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.872282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.872351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.872452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.872530 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.872708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.873051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.873218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.873295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.873457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.873533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.connection = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.873702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.873788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.873944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.874022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.874214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.874293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.874451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.874538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.874682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.874759 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.874915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.874990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.875147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.875223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.875383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.875461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.875613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.875689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.875847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.875922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.876088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.876166 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.876321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.876397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.876564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.876649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.876797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.876882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.877039 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.877115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.877266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.877342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.877502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.877587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.877733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.877809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.877906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80103) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 26 02:50:55.877983 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.878226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.878473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.878711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.878955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.879189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.879437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.879672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.879919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.880149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.880388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.880622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 26 02:50:55.880792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80102) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 26 02:50:55.908430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.908659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.908888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.910430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.910661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.910883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.917338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.920064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.945607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.945835 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.947575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:55.947810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.087747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.087984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.136357 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.136590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.138136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.138372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.200724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.200853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.200952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.201035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.201174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.201256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.201392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.201473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.201608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.201698 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.201821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.201902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.227101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.227232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.227329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.227394 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.227552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.227634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.227780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.227869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.227997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.228077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.228211 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.228290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.228424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.228516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.239534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.239632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.252469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.252659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.252761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.252893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.252974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.253115 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.253195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.253332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.253413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.253545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.253623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.253756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.253835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80102) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.253969 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80103) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 26 02:50:56.271541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.271752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 26 02:50:56.271964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.272087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.272181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 26 02:50:56.272288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 26 02:50:56.272490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.272571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.272690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 26 02:50:56.272803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.272955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.273056 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.273214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.273291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.273468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.273720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.284762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.285035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.285115 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.285296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.285376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.285568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.285648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.285845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.293824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.294117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.297142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.297383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.297464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.297659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.297740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.297938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.298019 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.298240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.298320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.298499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.298591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.298757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.298838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.299020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.299098 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.299282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.299367 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.299541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.299619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.299801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.299879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.300057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.326797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.326917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.328404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.328513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.328682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.328810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.334067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.334160 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.334337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.334418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.334617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.334698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.342518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.342609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.342752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.342842 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.347193 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.347284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.347476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.347556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.347754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.347835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.348030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.348110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.348298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.348377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.348566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.348646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.348826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.348904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.349086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.349174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.367740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.367829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.368011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.368092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.372667 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.373010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.376558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.376654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.376952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.377046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.381939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.382023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.385542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.385623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.389731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.389834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.390027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.390119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.390331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.390411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.390610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.390689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.390878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.390958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.391147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.391235 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.410609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.410769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.410886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.411047 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.411140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.411311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.411390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.411584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.419257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.419503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.420942 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.421212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.421298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.421483 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.424401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.424680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.424760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.424958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.425038 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.425231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.425322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.425546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.425625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.425819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.439281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.439554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.439636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.439836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.439916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.440109 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.440198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.440382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.441571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.441857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.441938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.442169 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.442251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.442465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.460430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.460891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.463707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.464185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.467037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.467545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.468771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.469266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.470417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.470690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.470921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.471002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.471196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.471481 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.478271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.478769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.479864 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.480372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.483116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.483411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.483631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.483799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.483920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.484070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.484300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.484574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.493090 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.493372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.493637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.493729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.493921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.494221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.502286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80103) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.502557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.502829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.502909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80102) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 26 02:50:56.503095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.503175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.503357 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.503437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.503654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.503733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.503923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.504011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.504248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.504337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.504520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.504599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.504791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.504916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.505055 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.505134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.505311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.505390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.505572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.505651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.505848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80103) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.505926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.506145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.506416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80102) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 26 02:50:56.543835 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fe39e9d9c0 pid: 80103 (default app) Jun 26 02:50:56.544518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fe39e9d9c0 pid: 80102 (default app) Jun 26 02:50:56.547190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd5690-54af-47de-a671-7c301eda9624 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:50:56.547479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd5690-54af-47de-a671-7c301eda9624 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:50:56.547754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd5690-54af-47de-a671-7c301eda9624 None None] PATH_INFO: `/v3` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:50:56.551949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Sun Jun 26 02:50:56 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:04.144241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e585efd-8fd5-4624-8217-a2debb37c44c None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:04.144800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e585efd-8fd5-4624-8217-a2debb37c44c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:04.144800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e585efd-8fd5-4624-8217-a2debb37c44c None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:04.149115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:51:04.155124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:04.155547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:04.155686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:04.223619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: INFO dbcounter [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] Registered counter for database keystone Jun 26 02:51:04.244781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG oslo_db.sqlalchemy.engines [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80103) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 26 02:51:04.251217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writer thread running {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 26 02:51:04.425142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG passlib.handlers.bcrypt [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80103) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 26 02:51:04.425500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG passlib.handlers.bcrypt [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80103) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 26 02:51:04.480233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:04.481879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:04.603071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 26 02:51:04.731681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e585efd-8fd5-4624-8217-a2debb37c44c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:04.805112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: INFO dbcounter [None req-4e585efd-8fd5-4624-8217-a2debb37c44c None None] Registered counter for database keystone Jun 26 02:51:04.825286 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG oslo_db.sqlalchemy.engines [None req-4e585efd-8fd5-4624-8217-a2debb37c44c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80102) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 26 02:51:04.831423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writer thread running {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 26 02:51:04.948953 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e585efd-8fd5-4624-8217-a2debb37c44c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:04.953785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6d7cd9-4c75-4513-b571-8071799437f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:04.954470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d7cd9-4c75-4513-b571-8071799437f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:04.954756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d7cd9-4c75-4513-b571-8071799437f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:04.955032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6d7cd9-4c75-4513-b571-8071799437f4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:04.963070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d7cd9-4c75-4513-b571-8071799437f4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:05.014920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bb6d7cd9-4c75-4513-b571-8071799437f4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 26 02:51:05.015206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6d7cd9-4c75-4513-b571-8071799437f4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:05.024658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Sun Jun 26 02:51:04 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 26 02:51:05.031079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:05.045803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a597502-3c0e-4e02-9fbb-522fcb0d7073 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:05.050093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c35953-d9bb-4421-a9b3-5a90dcec73f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:05.051215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c35953-d9bb-4421-a9b3-5a90dcec73f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:05.051500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c35953-d9bb-4421-a9b3-5a90dcec73f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:05.051772 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c35953-d9bb-4421-a9b3-5a90dcec73f8 admin admin] PATH_INFO: `/v3/projects/becb24f8723f432e9a3c4c740a191a0b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:05.055813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c35953-d9bb-4421-a9b3-5a90dcec73f8 admin admin] RBAC: Authorizing `identity:get_project(project_id=becb24f8723f432e9a3c4c740a191a0b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:05.105837 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d1c35953-d9bb-4421-a9b3-5a90dcec73f8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 26 02:51:05.106203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c35953-d9bb-4421-a9b3-5a90dcec73f8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:05.110737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Sun Jun 26 02:51:05 2022] GET /identity/v3/projects/becb24f8723f432e9a3c4c740a191a0b => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:07.079466 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7dbca-8a5b-48f9-b8b3-268ede127e85 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:07.079972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7dbca-8a5b-48f9-b8b3-268ede127e85 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:07.080072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a7dbca-8a5b-48f9-b8b3-268ede127e85 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:07.084195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:07.089347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338ebce-b3c6-4f39-bcf4-b8315b1b7064 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:07.089748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338ebce-b3c6-4f39-bcf4-b8315b1b7064 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:07.089907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a338ebce-b3c6-4f39-bcf4-b8315b1b7064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:07.109461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a338ebce-b3c6-4f39-bcf4-b8315b1b7064 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:07.110924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a338ebce-b3c6-4f39-bcf4-b8315b1b7064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:07.118601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 26 02:51:07.140537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7dbca-8a5b-48f9-b8b3-268ede127e85 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:07.152308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a7dbca-8a5b-48f9-b8b3-268ede127e85 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:07.156962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3084d9-1748-4b40-97de-5b2ea72b3142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:07.157586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3084d9-1748-4b40-97de-5b2ea72b3142 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:07.157870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3084d9-1748-4b40-97de-5b2ea72b3142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:07.158176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3084d9-1748-4b40-97de-5b2ea72b3142 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:07.165744 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3084d9-1748-4b40-97de-5b2ea72b3142 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:07.167554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3084d9-1748-4b40-97de-5b2ea72b3142 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:07.176776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Sun Jun 26 02:51:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 02:51:07.182025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338ebce-b3c6-4f39-bcf4-b8315b1b7064 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:07.190849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a338ebce-b3c6-4f39-bcf4-b8315b1b7064 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:07.195105 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b24a8-c6b8-4ca2-bdff-14aee5c0df05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:07.195949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b24a8-c6b8-4ca2-bdff-14aee5c0df05 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:07.196234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b24a8-c6b8-4ca2-bdff-14aee5c0df05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:07.196507 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b24a8-c6b8-4ca2-bdff-14aee5c0df05 admin admin] PATH_INFO: `/v3/users/a05c37bdf694480a8a14de0b18b2da4b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:07.197878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b24a8-c6b8-4ca2-bdff-14aee5c0df05 admin admin] RBAC: Authorizing `identity:get_user(user_id=a05c37bdf694480a8a14de0b18b2da4b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:07.199744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086b24a8-c6b8-4ca2-bdff-14aee5c0df05 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:07.201309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Sun Jun 26 02:51:07 2022] GET /identity/v3/users/a05c37bdf694480a8a14de0b18b2da4b => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:09.265192 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fdeb3-b8f1-4a0d-86f4-3d39cf7e3958 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:09.265695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fdeb3-b8f1-4a0d-86f4-3d39cf7e3958 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:09.265787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9fdeb3-b8f1-4a0d-86f4-3d39cf7e3958 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:09.267149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:09.272147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ab477-dfb5-4420-95a9-96bac2fdb6ad None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:09.272570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ab477-dfb5-4420-95a9-96bac2fdb6ad None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:09.272714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ab477-dfb5-4420-95a9-96bac2fdb6ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:09.289524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e71ab477-dfb5-4420-95a9-96bac2fdb6ad None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:09.290739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e71ab477-dfb5-4420-95a9-96bac2fdb6ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:09.369861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 6/11] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:09.392376 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8aeb9-344c-4a5e-97a6-49326ea570e3 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:09.392659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8aeb9-344c-4a5e-97a6-49326ea570e3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:09.392932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8aeb9-344c-4a5e-97a6-49326ea570e3 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:09.393047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ab477-dfb5-4420-95a9-96bac2fdb6ad None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:09.394337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 6/12] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:09.399101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b445b4d-7e9f-4f3d-b067-ae8f9dca2723 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:09.399396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b445b4d-7e9f-4f3d-b067-ae8f9dca2723 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:09.399679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b445b4d-7e9f-4f3d-b067-ae8f9dca2723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:09.416281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:09.439272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG passlib.handlers.bcrypt [None req-6b445b4d-7e9f-4f3d-b067-ae8f9dca2723 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80102) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 26 02:51:09.439700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG passlib.handlers.bcrypt [None req-6b445b4d-7e9f-4f3d-b067-ae8f9dca2723 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80102) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 26 02:51:09.485613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6b445b4d-7e9f-4f3d-b067-ae8f9dca2723 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:09.487497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6b445b4d-7e9f-4f3d-b067-ae8f9dca2723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:09.495693 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ab477-dfb5-4420-95a9-96bac2fdb6ad None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:09.529241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb84e80-bcbe-4b12-9d1f-650e890cc621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:09.529961 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb84e80-bcbe-4b12-9d1f-650e890cc621 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:09.530282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb84e80-bcbe-4b12-9d1f-650e890cc621 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:09.530567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb84e80-bcbe-4b12-9d1f-650e890cc621 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:09.541449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb84e80-bcbe-4b12-9d1f-650e890cc621 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-8eb84e80-bcbe-4b12-9d1f-650e890cc621 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:09.553271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:09.554170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:09.554970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 7/13] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:51:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:09.560388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb84e80-bcbe-4b12-9d1f-650e890cc621 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:09.586012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 7/14] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:09.608555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b445b4d-7e9f-4f3d-b067-ae8f9dca2723 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:09.641172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:09.659716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb84e80-bcbe-4b12-9d1f-650e890cc621 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:09.692146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827725f-f1d1-4774-ad36-5da1714f2516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:09.692813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827725f-f1d1-4774-ad36-5da1714f2516 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:09.693102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827725f-f1d1-4774-ad36-5da1714f2516 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:09.693380 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4827725f-f1d1-4774-ad36-5da1714f2516 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:09.694158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827725f-f1d1-4774-ad36-5da1714f2516 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:09.694470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827725f-f1d1-4774-ad36-5da1714f2516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:09.695850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4827725f-f1d1-4774-ad36-5da1714f2516 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:09.708026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 8/15] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:09.708155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b445b4d-7e9f-4f3d-b067-ae8f9dca2723 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:09.710380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:09.713465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827725f-f1d1-4774-ad36-5da1714f2516 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:09.746116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51704a2e-2b4d-4975-b012-94411e556c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:09.747044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51704a2e-2b4d-4975-b012-94411e556c09 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:09.747370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51704a2e-2b4d-4975-b012-94411e556c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:09.747650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51704a2e-2b4d-4975-b012-94411e556c09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:09.811803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4827725f-f1d1-4774-ad36-5da1714f2516 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:09.813421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51704a2e-2b4d-4975-b012-94411e556c09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:09.814954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51704a2e-2b4d-4975-b012-94411e556c09 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:09.845402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01b081-5b5d-4d08-8e14-4d6db09688f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:09.846314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01b081-5b5d-4d08-8e14-4d6db09688f0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:09.846604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01b081-5b5d-4d08-8e14-4d6db09688f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:09.846877 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb01b081-5b5d-4d08-8e14-4d6db09688f0 None admin] PATH_INFO: `/v3/users/a05c37bdf694480a8a14de0b18b2da4b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:09.848130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01b081-5b5d-4d08-8e14-4d6db09688f0 None admin] RBAC: Authorizing `identity:get_user(user_id=a05c37bdf694480a8a14de0b18b2da4b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:09.849589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb01b081-5b5d-4d08-8e14-4d6db09688f0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:09.851307 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 9/16] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:09 2022] GET /identity/v3/users/a05c37bdf694480a8a14de0b18b2da4b => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:09.856656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01b081-5b5d-4d08-8e14-4d6db09688f0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:09.871127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:09.913464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 8/17] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 26 02:51:09.919281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51704a2e-2b4d-4975-b012-94411e556c09 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:09.922063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:09.957447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb01b081-5b5d-4d08-8e14-4d6db09688f0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:09.992542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65076db-5e0f-4fe7-a23b-422cf09ecad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:09.993317 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65076db-5e0f-4fe7-a23b-422cf09ecad4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:09.993602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65076db-5e0f-4fe7-a23b-422cf09ecad4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:09.993875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65076db-5e0f-4fe7-a23b-422cf09ecad4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:09.995081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65076db-5e0f-4fe7-a23b-422cf09ecad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:09.997007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65076db-5e0f-4fe7-a23b-422cf09ecad4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:09.998916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 10/18] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:10.004763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65076db-5e0f-4fe7-a23b-422cf09ecad4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:10.018846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51704a2e-2b4d-4975-b012-94411e556c09 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:10.051198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6f6ef-ab4d-4d87-8778-df7d0b40a192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:10.051982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6f6ef-ab4d-4d87-8778-df7d0b40a192 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:10.052262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6f6ef-ab4d-4d87-8778-df7d0b40a192 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:10.052542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6f6ef-ab4d-4d87-8778-df7d0b40a192 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:10.070952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6f6ef-ab4d-4d87-8778-df7d0b40a192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:10.072340 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6f6ef-ab4d-4d87-8778-df7d0b40a192 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:10.082730 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-4df6f6ef-ab4d-4d87-8778-df7d0b40a192 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:51:10.095664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:10.096489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:10.097269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:51:10.097269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:51:10.097269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:51:10.097269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:10.097269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 9/19] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:10.102709 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6f6ef-ab4d-4d87-8778-df7d0b40a192 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:10.102829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65076db-5e0f-4fe7-a23b-422cf09ecad4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:10.136396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3d6b44-7102-4657-a037-0c7011efc1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:10.137143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d6b44-7102-4657-a037-0c7011efc1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:10.138018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d6b44-7102-4657-a037-0c7011efc1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:10.138341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3d6b44-7102-4657-a037-0c7011efc1e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:10.139189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d6b44-7102-4657-a037-0c7011efc1e6 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.domain.id=default, user.id=a05c37bdf694480a8a14de0b18b2da4b) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:10.139500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d6b44-7102-4657-a037-0c7011efc1e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:10.140962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3d6b44-7102-4657-a037-0c7011efc1e6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:10.147541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 11/20] 10.222.0.13 () {56 vars in 1322 bytes} [Sun Jun 26 02:51:10 2022] GET /identity/v3/role_assignments?user.id=a05c37bdf694480a8a14de0b18b2da4b&scope.domain.id=default&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:10.160304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:10.210178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6f6ef-ab4d-4d87-8778-df7d0b40a192 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:10.244927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4f7522-3ab5-413d-8af5-c8aba406be3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:10.245610 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f7522-3ab5-413d-8af5-c8aba406be3e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:10.245900 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f7522-3ab5-413d-8af5-c8aba406be3e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:10.246219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f7522-3ab5-413d-8af5-c8aba406be3e None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:10.247493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f7522-3ab5-413d-8af5-c8aba406be3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:10.247808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f7522-3ab5-413d-8af5-c8aba406be3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:10.249511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4f7522-3ab5-413d-8af5-c8aba406be3e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:10.268835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 10/21] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:12.728484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0509aed-bc3e-49ec-8848-6fc8c3156f6d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:12.729367 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0509aed-bc3e-49ec-8848-6fc8c3156f6d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:12.729797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0509aed-bc3e-49ec-8848-6fc8c3156f6d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:12.731421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 12/22] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:12.737208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd740b-780d-43b3-8874-c771fd65ff16 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:12.737665 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd740b-780d-43b3-8874-c771fd65ff16 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:12.737980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd740b-780d-43b3-8874-c771fd65ff16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:12.763510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3bfd740b-780d-43b3-8874-c771fd65ff16 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:12.764755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3bfd740b-780d-43b3-8874-c771fd65ff16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:12.781722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f724d-91af-48a2-887c-05221d5d03a3 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:12.782025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f724d-91af-48a2-887c-05221d5d03a3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:12.782324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f724d-91af-48a2-887c-05221d5d03a3 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:12.784093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 13/23] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:12.790412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998d09b-6f97-41ce-8be0-302f0d79eeec None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:12.790709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998d09b-6f97-41ce-8be0-302f0d79eeec None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:12.790996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998d09b-6f97-41ce-8be0-302f0d79eeec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:12.832867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a998d09b-6f97-41ce-8be0-302f0d79eeec None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:12.832867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a998d09b-6f97-41ce-8be0-302f0d79eeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:12.900204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 11/24] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:12.901418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88dacb-9a15-49e4-96ea-36454b90b833 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:12.901708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88dacb-9a15-49e4-96ea-36454b90b833 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:12.901982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88dacb-9a15-49e4-96ea-36454b90b833 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:12.903916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 12/25] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:12.911372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef902e5-3685-4a46-8d27-f6e1756c318d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:12.911661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef902e5-3685-4a46-8d27-f6e1756c318d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:12.911941 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef902e5-3685-4a46-8d27-f6e1756c318d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:12.936736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 14/26] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:12.943478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5ef902e5-3685-4a46-8d27-f6e1756c318d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:12.944674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5ef902e5-3685-4a46-8d27-f6e1756c318d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:12.968238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998d09b-6f97-41ce-8be0-302f0d79eeec None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:13.015774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:13.043498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 13/27] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:13.046117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef902e5-3685-4a46-8d27-f6e1756c318d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:13.072188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:13.107904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998d09b-6f97-41ce-8be0-302f0d79eeec None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:13.153854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acee15-740d-4ba8-bd9c-48dd39b2da4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:13.155388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acee15-740d-4ba8-bd9c-48dd39b2da4e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.156326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acee15-740d-4ba8-bd9c-48dd39b2da4e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.157054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acee15-740d-4ba8-bd9c-48dd39b2da4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:13.171578 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acee15-740d-4ba8-bd9c-48dd39b2da4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:13.174221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3acee15-740d-4ba8-bd9c-48dd39b2da4e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:13.175864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef902e5-3685-4a46-8d27-f6e1756c318d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f3acee15-740d-4ba8-bd9c-48dd39b2da4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:51:13.196392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:13.197233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:13.197997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:51:13.197997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:51:13.197997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:51:13.197997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:13.200308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 15/28] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:13.202074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc88897-5089-4545-bc09-020a0ae0bdb0 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.202545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc88897-5089-4545-bc09-020a0ae0bdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.203006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc88897-5089-4545-bc09-020a0ae0bdb0 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:13.204530 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 16/29] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:13.209135 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc88897-5089-4545-bc09-020a0ae0bdb0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:13.219330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948276e4-17d6-48fe-ad89-cb4eac22c936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:13.220117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948276e4-17d6-48fe-ad89-cb4eac22c936 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.220478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948276e4-17d6-48fe-ad89-cb4eac22c936 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.220828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948276e4-17d6-48fe-ad89-cb4eac22c936 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:13.221658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948276e4-17d6-48fe-ad89-cb4eac22c936 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:13.223546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948276e4-17d6-48fe-ad89-cb4eac22c936 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:13.328213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc88897-5089-4545-bc09-020a0ae0bdb0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:13.358545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:13.368662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c07c50-d54d-498b-afd4-b132fd4720e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:13.369740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c07c50-d54d-498b-afd4-b132fd4720e2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.370255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c07c50-d54d-498b-afd4-b132fd4720e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.370681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c07c50-d54d-498b-afd4-b132fd4720e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:13.382276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c07c50-d54d-498b-afd4-b132fd4720e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f4c07c50-d54d-498b-afd4-b132fd4720e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:13.390892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:13.391931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:13.392880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:13.394999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 17/30] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:51:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:13.395968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c07c50-d54d-498b-afd4-b132fd4720e2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:13.410170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 14/31] 10.222.0.13 () {60 vars in 1131 bytes} [Sun Jun 26 02:51:13 2022] POST /identity/v3/roles => generated 219 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:51:13.412198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d582c97-9aee-4009-990d-dc7bfd5e0346 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.412686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d582c97-9aee-4009-990d-dc7bfd5e0346 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.413133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d582c97-9aee-4009-990d-dc7bfd5e0346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:13.451155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7d582c97-9aee-4009-990d-dc7bfd5e0346 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:13.455574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7d582c97-9aee-4009-990d-dc7bfd5e0346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:13.508074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:13.587759 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 15/32] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:13.589757 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d582c97-9aee-4009-990d-dc7bfd5e0346 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:13.595192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c07c50-d54d-498b-afd4-b132fd4720e2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:13.653451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d381c40-e7f4-4c6c-8fe9-fd73b2273b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:13.654468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d381c40-e7f4-4c6c-8fe9-fd73b2273b6f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.654863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d381c40-e7f4-4c6c-8fe9-fd73b2273b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.655257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d381c40-e7f4-4c6c-8fe9-fd73b2273b6f None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:13.656347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d381c40-e7f4-4c6c-8fe9-fd73b2273b6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:13.656985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d381c40-e7f4-4c6c-8fe9-fd73b2273b6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:13.659153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d381c40-e7f4-4c6c-8fe9-fd73b2273b6f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:13.679919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 18/33] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:13.680938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d381c40-e7f4-4c6c-8fe9-fd73b2273b6f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:13.709791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d582c97-9aee-4009-990d-dc7bfd5e0346 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:13.734842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:13.750365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a85fc3-2d67-4b5d-a039-be1087331a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:13.751138 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a85fc3-2d67-4b5d-a039-be1087331a41 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.751433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a85fc3-2d67-4b5d-a039-be1087331a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.751856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a85fc3-2d67-4b5d-a039-be1087331a41 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:13.752623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a85fc3-2d67-4b5d-a039-be1087331a41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:13.752937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a85fc3-2d67-4b5d-a039-be1087331a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:13.754485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a85fc3-2d67-4b5d-a039-be1087331a41 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:13.768797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 16/34] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:13.769529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a85fc3-2d67-4b5d-a039-be1087331a41 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:13.792772 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d381c40-e7f4-4c6c-8fe9-fd73b2273b6f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:13.805891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:13.830724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806ef1e-81cc-4e97-a2d4-c12c3e776668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:13.831406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806ef1e-81cc-4e97-a2d4-c12c3e776668 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.831785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806ef1e-81cc-4e97-a2d4-c12c3e776668 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.832067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806ef1e-81cc-4e97-a2d4-c12c3e776668 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:13.832826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806ef1e-81cc-4e97-a2d4-c12c3e776668 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:13.834240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806ef1e-81cc-4e97-a2d4-c12c3e776668 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:13.889550 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a85fc3-2d67-4b5d-a039-be1087331a41 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:13.925135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4ac82-a12e-4297-9f80-53cc97207fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:13.925809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4ac82-a12e-4297-9f80-53cc97207fae None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.926123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4ac82-a12e-4297-9f80-53cc97207fae None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.926407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4ac82-a12e-4297-9f80-53cc97207fae None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:13.927363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4ac82-a12e-4297-9f80-53cc97207fae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:13.928807 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4ac82-a12e-4297-9f80-53cc97207fae None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:13.957610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 19/35] 10.222.0.13 () {60 vars in 1131 bytes} [Sun Jun 26 02:51:13 2022] POST /identity/v3/roles => generated 225 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:51:13.958672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806ef1e-81cc-4e97-a2d4-c12c3e776668 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:13.959467 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 17/36] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:51:13 2022] POST /identity/v3/projects => generated 301 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:51:13.961110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5db4b-dee0-4bd9-b4e8-c7a8d988994d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.961629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5db4b-dee0-4bd9-b4e8-c7a8d988994d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.962168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e5db4b-dee0-4bd9-b4e8-c7a8d988994d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:13.963930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 18/37] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:13.974268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e57b4-f11f-4df7-a26c-93bf25c1547c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:13.975057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e57b4-f11f-4df7-a26c-93bf25c1547c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:13.975580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e57b4-f11f-4df7-a26c-93bf25c1547c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:14.008830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9e4e57b4-f11f-4df7-a26c-93bf25c1547c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:14.014116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9e4e57b4-f11f-4df7-a26c-93bf25c1547c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:14.160133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 19/38] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:14.178554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806ef1e-81cc-4e97-a2d4-c12c3e776668 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:14.183231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e57b4-f11f-4df7-a26c-93bf25c1547c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:14.187459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:14.201149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:14.229300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3615c-27d8-49b0-ad12-613fe4db1ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:14.230549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3615c-27d8-49b0-ad12-613fe4db1ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:14.231074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3615c-27d8-49b0-ad12-613fe4db1ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:14.231571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3615c-27d8-49b0-ad12-613fe4db1ba7 None admin] PATH_INFO: `/v3/users/a05c37bdf694480a8a14de0b18b2da4b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:14.233133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3615c-27d8-49b0-ad12-613fe4db1ba7 None admin] RBAC: Authorizing `identity:get_user(user_id=a05c37bdf694480a8a14de0b18b2da4b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:14.234974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3615c-27d8-49b0-ad12-613fe4db1ba7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:14.238276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 20/39] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:13 2022] GET /identity/v3/users/a05c37bdf694480a8a14de0b18b2da4b => generated 247 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:14.243837 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3615c-27d8-49b0-ad12-613fe4db1ba7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:14.314334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e57b4-f11f-4df7-a26c-93bf25c1547c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:14.349875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1ffd9-8dcd-4529-b38d-5e5b8e3f6eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:14.350568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1ffd9-8dcd-4529-b38d-5e5b8e3f6eb7 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:14.350850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1ffd9-8dcd-4529-b38d-5e5b8e3f6eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:14.351124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1ffd9-8dcd-4529-b38d-5e5b8e3f6eb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:14.351874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1ffd9-8dcd-4529-b38d-5e5b8e3f6eb7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:14.353274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1ffd9-8dcd-4529-b38d-5e5b8e3f6eb7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:14.357676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3615c-27d8-49b0-ad12-613fe4db1ba7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:14.366185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 20/40] 10.222.0.13 () {60 vars in 1131 bytes} [Sun Jun 26 02:51:14 2022] POST /identity/v3/roles => generated 223 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:51:14.367410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236fd86-9eb4-47c4-9380-ad2bea25dbd6 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:14.367700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236fd86-9eb4-47c4-9380-ad2bea25dbd6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:14.367975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236fd86-9eb4-47c4-9380-ad2bea25dbd6 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:14.369246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 21/41] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:14.379837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed5fd8-1951-4900-8423-33444d8ec7a4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:14.380131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed5fd8-1951-4900-8423-33444d8ec7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:14.380436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ed5fd8-1951-4900-8423-33444d8ec7a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:14.399484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-46ed5fd8-1951-4900-8423-33444d8ec7a4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:14.400732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-46ed5fd8-1951-4900-8423-33444d8ec7a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:14.402209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b6364-7a7b-43f6-aecf-67afa4306486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:14.403677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b6364-7a7b-43f6-aecf-67afa4306486 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:14.403964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b6364-7a7b-43f6-aecf-67afa4306486 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:14.404245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b6364-7a7b-43f6-aecf-67afa4306486 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:14.406381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b6364-7a7b-43f6-aecf-67afa4306486 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:14.407855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b6364-7a7b-43f6-aecf-67afa4306486 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:14.414447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 21/42] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:14.415563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b6364-7a7b-43f6-aecf-67afa4306486 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:14.466021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:14.480480 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:14.486100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 22/43] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:14.511453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed5fd8-1951-4900-8423-33444d8ec7a4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:14.628922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ed5fd8-1951-4900-8423-33444d8ec7a4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:14.650810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b6364-7a7b-43f6-aecf-67afa4306486 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:14.663350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7107b-1cfa-42b4-9bcb-16d5139010f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:14.664147 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7107b-1cfa-42b4-9bcb-16d5139010f1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:14.664436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7107b-1cfa-42b4-9bcb-16d5139010f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:14.664766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7107b-1cfa-42b4-9bcb-16d5139010f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:14.665889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7107b-1cfa-42b4-9bcb-16d5139010f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:14.667366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7107b-1cfa-42b4-9bcb-16d5139010f1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:14.669645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 23/44] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:14.675262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7107b-1cfa-42b4-9bcb-16d5139010f1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:14.688273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdecd6c-ff4b-4a63-aa3b-bd5224b45904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:14.689187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdecd6c-ff4b-4a63-aa3b-bd5224b45904 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:14.689541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdecd6c-ff4b-4a63-aa3b-bd5224b45904 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:14.689971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdecd6c-ff4b-4a63-aa3b-bd5224b45904 None admin] PATH_INFO: `/v3/domains/default/users/a05c37bdf694480a8a14de0b18b2da4b/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:14.692265 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdecd6c-ff4b-4a63-aa3b-bd5224b45904 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=a05c37bdf694480a8a14de0b18b2da4b, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:14.694195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdecd6c-ff4b-4a63-aa3b-bd5224b45904 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:14.708046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 22/45] 10.222.0.13 () {58 vars in 1361 bytes} [Sun Jun 26 02:51:14 2022] PUT /identity/v3/domains/default/users/a05c37bdf694480a8a14de0b18b2da4b/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 02:51:14.743648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:14.779711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7107b-1cfa-42b4-9bcb-16d5139010f1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:14.815219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b181f9e-996f-4764-8fe7-d85d40d9e3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:14.815901 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b181f9e-996f-4764-8fe7-d85d40d9e3eb None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:14.816188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b181f9e-996f-4764-8fe7-d85d40d9e3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:14.816468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b181f9e-996f-4764-8fe7-d85d40d9e3eb None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:14.817199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b181f9e-996f-4764-8fe7-d85d40d9e3eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:14.818715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b181f9e-996f-4764-8fe7-d85d40d9e3eb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:14.853756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 24/46] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:51:14 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:51:16.745847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789da8d1-8a89-4b00-8b3e-a20dcd7142c5 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:16.746403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789da8d1-8a89-4b00-8b3e-a20dcd7142c5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:16.746504 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789da8d1-8a89-4b00-8b3e-a20dcd7142c5 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:16.747915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 23/47] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:16.753438 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473eecb-23a3-429b-8702-5cfabf75b0c7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:16.753849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473eecb-23a3-429b-8702-5cfabf75b0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:16.754157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473eecb-23a3-429b-8702-5cfabf75b0c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:16.770815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6473eecb-23a3-429b-8702-5cfabf75b0c7 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:16.772012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6473eecb-23a3-429b-8702-5cfabf75b0c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:16.837218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 25/48] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:16.860556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a263b3-956e-40b5-b5b2-8738a51cbb80 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:16.860823 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473eecb-23a3-429b-8702-5cfabf75b0c7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:16.860920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a263b3-956e-40b5-b5b2-8738a51cbb80 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:16.861113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a263b3-956e-40b5-b5b2-8738a51cbb80 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:16.862547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 24/49] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:16.867326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460b73f-db94-484d-a812-3a1a9e9f4dd2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:16.867611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460b73f-db94-484d-a812-3a1a9e9f4dd2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:16.867894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460b73f-db94-484d-a812-3a1a9e9f4dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:16.884315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e460b73f-db94-484d-a812-3a1a9e9f4dd2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:16.885817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e460b73f-db94-484d-a812-3a1a9e9f4dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:16.910791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:16.954886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 25/50] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:16.961328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473eecb-23a3-429b-8702-5cfabf75b0c7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:16.977628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460b73f-db94-484d-a812-3a1a9e9f4dd2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:16.995600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae068837-8934-46d0-ac58-5b62551d2f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:16.996967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae068837-8934-46d0-ac58-5b62551d2f0d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:16.997311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae068837-8934-46d0-ac58-5b62551d2f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:16.997653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae068837-8934-46d0-ac58-5b62551d2f0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:17.007615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:17.009204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae068837-8934-46d0-ac58-5b62551d2f0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ae068837-8934-46d0-ac58-5b62551d2f0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:17.018930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:17.019747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:17.020486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 26/51] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:51:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:17.025874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae068837-8934-46d0-ac58-5b62551d2f0d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:17.077089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460b73f-db94-484d-a812-3a1a9e9f4dd2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:17.110792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cc0d6-e366-4404-8269-a96e2fd3d389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:17.111613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cc0d6-e366-4404-8269-a96e2fd3d389 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:17.111907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cc0d6-e366-4404-8269-a96e2fd3d389 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:17.112188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033cc0d6-e366-4404-8269-a96e2fd3d389 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:17.113451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cc0d6-e366-4404-8269-a96e2fd3d389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:17.115028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033cc0d6-e366-4404-8269-a96e2fd3d389 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:17.116782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 26/52] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:17.123359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cc0d6-e366-4404-8269-a96e2fd3d389 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:17.222588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:17.225768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033cc0d6-e366-4404-8269-a96e2fd3d389 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:17.229504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae068837-8934-46d0-ac58-5b62551d2f0d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:17.241137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:17.261268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531e3bb-c017-4033-a269-1e1de1695eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:17.261867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e3bb-c017-4033-a269-1e1de1695eb4 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:17.262192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e3bb-c017-4033-a269-1e1de1695eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:17.262478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4531e3bb-c017-4033-a269-1e1de1695eb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:17.263198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531e3bb-c017-4033-a269-1e1de1695eb4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:17.264654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4531e3bb-c017-4033-a269-1e1de1695eb4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:17.265677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e0644-07b3-4751-b1cc-d9e2da61b894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:17.266371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e0644-07b3-4751-b1cc-d9e2da61b894 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:17.266655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e0644-07b3-4751-b1cc-d9e2da61b894 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:17.266931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861e0644-07b3-4751-b1cc-d9e2da61b894 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:17.267679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e0644-07b3-4751-b1cc-d9e2da61b894 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:17.267985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e0644-07b3-4751-b1cc-d9e2da61b894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:17.269353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861e0644-07b3-4751-b1cc-d9e2da61b894 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:17.277615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 27/53] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:17.283408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e0644-07b3-4751-b1cc-d9e2da61b894 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:17.291089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 27/54] 10.222.0.13 () {60 vars in 1140 bytes} [Sun Jun 26 02:51:17 2022] POST /identity/v3/projects => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:51:17.388124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861e0644-07b3-4751-b1cc-d9e2da61b894 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:17.424235 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30c8bf-34b0-41c7-81eb-154f138d6b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:17.425128 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30c8bf-34b0-41c7-81eb-154f138d6b75 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:17.425410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30c8bf-34b0-41c7-81eb-154f138d6b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:17.425681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30c8bf-34b0-41c7-81eb-154f138d6b75 None admin] PATH_INFO: `/v3/users/a05c37bdf694480a8a14de0b18b2da4b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:17.427200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30c8bf-34b0-41c7-81eb-154f138d6b75 None admin] RBAC: Authorizing `identity:get_user(user_id=a05c37bdf694480a8a14de0b18b2da4b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:17.428652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30c8bf-34b0-41c7-81eb-154f138d6b75 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:17.430360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 28/55] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:17 2022] GET /identity/v3/users/a05c37bdf694480a8a14de0b18b2da4b => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:17.436296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531e3bb-c017-4033-a269-1e1de1695eb4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:17.536660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4531e3bb-c017-4033-a269-1e1de1695eb4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:17.569110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b1459-a9ec-44aa-8e51-fa6e013356dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:17.569896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b1459-a9ec-44aa-8e51-fa6e013356dd None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:17.570215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b1459-a9ec-44aa-8e51-fa6e013356dd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:17.570494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b1459-a9ec-44aa-8e51-fa6e013356dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:17.571648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b1459-a9ec-44aa-8e51-fa6e013356dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:17.573103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b1459-a9ec-44aa-8e51-fa6e013356dd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:17.575334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 28/56] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:17.581113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30c8bf-34b0-41c7-81eb-154f138d6b75 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:17.627565 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:17.683442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30c8bf-34b0-41c7-81eb-154f138d6b75 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:17.715871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529411c-bb9f-490d-97d5-fd7a5a2ea3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:17.716524 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529411c-bb9f-490d-97d5-fd7a5a2ea3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:17.716809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529411c-bb9f-490d-97d5-fd7a5a2ea3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:17.717088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529411c-bb9f-490d-97d5-fd7a5a2ea3d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:17.717913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529411c-bb9f-490d-97d5-fd7a5a2ea3d5 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.domain.id=default, user.id=a05c37bdf694480a8a14de0b18b2da4b) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:17.718257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529411c-bb9f-490d-97d5-fd7a5a2ea3d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:17.719682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529411c-bb9f-490d-97d5-fd7a5a2ea3d5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:17.728616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 29/57] 10.222.0.13 () {56 vars in 1322 bytes} [Sun Jun 26 02:51:17 2022] GET /identity/v3/role_assignments?user.id=a05c37bdf694480a8a14de0b18b2da4b&scope.domain.id=default&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:19.312625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0cf32-fc31-4a3d-9fbb-85b0b961e522 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:19.313131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0cf32-fc31-4a3d-9fbb-85b0b961e522 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:19.313213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0cf32-fc31-4a3d-9fbb-85b0b961e522 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:19.314628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 29/58] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:19.319586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357aafe-1bea-4ead-9815-839a655a6457 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:19.319942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357aafe-1bea-4ead-9815-839a655a6457 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:19.320153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357aafe-1bea-4ead-9815-839a655a6457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:19.336344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a357aafe-1bea-4ead-9815-839a655a6457 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:19.337532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a357aafe-1bea-4ead-9815-839a655a6457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:19.402021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 30/59] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:19.426243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0cf32-fc31-4a3d-9fbb-85b0b961e522 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:19.448119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:19.526286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0cf32-fc31-4a3d-9fbb-85b0b961e522 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:19.558472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59253c61-667d-4ee5-b937-7f889f32528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:19.559281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253c61-667d-4ee5-b937-7f889f32528f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:19.559569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253c61-667d-4ee5-b937-7f889f32528f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:19.559848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59253c61-667d-4ee5-b937-7f889f32528f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:19.561020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59253c61-667d-4ee5-b937-7f889f32528f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:19.562499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59253c61-667d-4ee5-b937-7f889f32528f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:19.564197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 30/60] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:19.569546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357aafe-1bea-4ead-9815-839a655a6457 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:19.602743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:19.667142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357aafe-1bea-4ead-9815-839a655a6457 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:19.699361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b7d8f0-3265-4625-b00d-6b80047af703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:19.700581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7d8f0-3265-4625-b00d-6b80047af703 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:19.700866 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7d8f0-3265-4625-b00d-6b80047af703 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:19.701144 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b7d8f0-3265-4625-b00d-6b80047af703 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:19.701857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7d8f0-3265-4625-b00d-6b80047af703 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:19.703326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b7d8f0-3265-4625-b00d-6b80047af703 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:19.727963 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 31/61] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:51:19 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:51:22.651373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52fad9-54e3-4de6-bcf3-c26eb000a78d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:22.651874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52fad9-54e3-4de6-bcf3-c26eb000a78d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:22.651975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52fad9-54e3-4de6-bcf3-c26eb000a78d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:22.654002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 31/62] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:22.659515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1e185-cf69-48ba-b964-1f267101d31d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:22.660356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1e185-cf69-48ba-b964-1f267101d31d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:22.660889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1e185-cf69-48ba-b964-1f267101d31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:22.681415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dbc1e185-cf69-48ba-b964-1f267101d31d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:22.682641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dbc1e185-cf69-48ba-b964-1f267101d31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:22.768350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 32/63] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:22.807966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52fad9-54e3-4de6-bcf3-c26eb000a78d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:22.930520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:22.941843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52fad9-54e3-4de6-bcf3-c26eb000a78d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:22.989517 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2761a556-962c-4536-ae81-03cb004d931f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:22.990350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761a556-962c-4536-ae81-03cb004d931f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:22.990646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761a556-962c-4536-ae81-03cb004d931f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:22.990932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2761a556-962c-4536-ae81-03cb004d931f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.007174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2761a556-962c-4536-ae81-03cb004d931f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-2761a556-962c-4536-ae81-03cb004d931f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:23.015091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:23.015945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:23.016672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 32/64] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:23.022463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1e185-cf69-48ba-b964-1f267101d31d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:23.054382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:23.077257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f3ed0-40d1-49e9-b2e8-33cb81a2e8d0 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.077973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f3ed0-40d1-49e9-b2e8-33cb81a2e8d0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.078513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f3ed0-40d1-49e9-b2e8-33cb81a2e8d0 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.080749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 33/65] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:51:23.088485 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566050b7-f76c-41cf-952d-52549d410c3a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.089041 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566050b7-f76c-41cf-952d-52549d410c3a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.089551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566050b7-f76c-41cf-952d-52549d410c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.117607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-566050b7-f76c-41cf-952d-52549d410c3a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:23.120180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-566050b7-f76c-41cf-952d-52549d410c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:23.157317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1e185-cf69-48ba-b964-1f267101d31d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:23.200492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131f6af-7a1c-4d71-8d33-5c864d96b433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:23.201871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131f6af-7a1c-4d71-8d33-5c864d96b433 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.202369 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131f6af-7a1c-4d71-8d33-5c864d96b433 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.203093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f131f6af-7a1c-4d71-8d33-5c864d96b433 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.204035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131f6af-7a1c-4d71-8d33-5c864d96b433 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:23.204528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131f6af-7a1c-4d71-8d33-5c864d96b433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:23.206169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f131f6af-7a1c-4d71-8d33-5c864d96b433 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:23.206542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 34/66] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:23.208212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e73c5a-8d04-42ef-8a06-6f58f6a44c3a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.208725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e73c5a-8d04-42ef-8a06-6f58f6a44c3a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.209208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e73c5a-8d04-42ef-8a06-6f58f6a44c3a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.210797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 35/67] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:23.223440 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cdc06-69dc-45b1-b417-5610d77aafb7 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.224018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cdc06-69dc-45b1-b417-5610d77aafb7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.224515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cdc06-69dc-45b1-b417-5610d77aafb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.233043 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 33/68] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:23.239056 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131f6af-7a1c-4d71-8d33-5c864d96b433 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:23.249464 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c43cdc06-69dc-45b1-b417-5610d77aafb7 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:23.250887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c43cdc06-69dc-45b1-b417-5610d77aafb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:23.334068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 36/69] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 26 02:51:23.334068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cdc06-69dc-45b1-b417-5610d77aafb7 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:23.335934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:23.344867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:23.367635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f131f6af-7a1c-4d71-8d33-5c864d96b433 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:23.405916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453cdaf-e6ca-4250-b845-f0efd53f9d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:23.406842 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453cdaf-e6ca-4250-b845-f0efd53f9d3a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.407130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453cdaf-e6ca-4250-b845-f0efd53f9d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.407413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453cdaf-e6ca-4250-b845-f0efd53f9d3a None admin] PATH_INFO: `/v3/users/19c972280f644c03b032e16fcb039658` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.424403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453cdaf-e6ca-4250-b845-f0efd53f9d3a None admin] RBAC: Authorizing `identity:get_user(user_id=19c972280f644c03b032e16fcb039658)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:23.425874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453cdaf-e6ca-4250-b845-f0efd53f9d3a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:23.430201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 34/70] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:23 2022] GET /identity/v3/users/19c972280f644c03b032e16fcb039658 => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:23.434894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919e5e4-14f3-4cb9-bc8e-400b4e434d76 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.434894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919e5e4-14f3-4cb9-bc8e-400b4e434d76 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.434894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919e5e4-14f3-4cb9-bc8e-400b4e434d76 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.434894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 35/71] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:23.434894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919e5e4-14f3-4cb9-bc8e-400b4e434d76 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:23.439455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cdc06-69dc-45b1-b417-5610d77aafb7 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:23.473956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81f7dd-b07b-4229-a58c-8a379e2a357c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:23.474723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81f7dd-b07b-4229-a58c-8a379e2a357c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.475010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81f7dd-b07b-4229-a58c-8a379e2a357c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.475287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c81f7dd-b07b-4229-a58c-8a379e2a357c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.486309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c81f7dd-b07b-4229-a58c-8a379e2a357c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-4c81f7dd-b07b-4229-a58c-8a379e2a357c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:23.493858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:23.494724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:23.495452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 37/72] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:51:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:23.496047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81f7dd-b07b-4229-a58c-8a379e2a357c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:23.548014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919e5e4-14f3-4cb9-bc8e-400b4e434d76 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:23.567400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:23.583213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf79cc0-013a-4e3e-b0aa-6148a9548e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:23.584026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf79cc0-013a-4e3e-b0aa-6148a9548e7e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.584311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf79cc0-013a-4e3e-b0aa-6148a9548e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.584586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf79cc0-013a-4e3e-b0aa-6148a9548e7e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.593336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:23.596818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf79cc0-013a-4e3e-b0aa-6148a9548e7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:23.603727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c81f7dd-b07b-4229-a58c-8a379e2a357c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-fcf79cc0-013a-4e3e-b0aa-6148a9548e7e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:23.606779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:23.607682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:23.608600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:23.610614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 36/73] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:51:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:23.611949 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c707c14-a886-4fc0-9145-58efa602806c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.612358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c707c14-a886-4fc0-9145-58efa602806c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.612753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c707c14-a886-4fc0-9145-58efa602806c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.630097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8c707c14-a886-4fc0-9145-58efa602806c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:23.631283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8c707c14-a886-4fc0-9145-58efa602806c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:23.642972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1e538-cde4-401e-9160-34ef5f5e459a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:23.643629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1e538-cde4-401e-9160-34ef5f5e459a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.643922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1e538-cde4-401e-9160-34ef5f5e459a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.644213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1e538-cde4-401e-9160-34ef5f5e459a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.697068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 37/74] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:23.697793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c707c14-a886-4fc0-9145-58efa602806c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:23.713102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1e538-cde4-401e-9160-34ef5f5e459a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:23.714639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1e538-cde4-401e-9160-34ef5f5e459a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:23.780292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:23.804469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c707c14-a886-4fc0-9145-58efa602806c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:23.819435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 38/75] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:23.819435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1e538-cde4-401e-9160-34ef5f5e459a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:23.834729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:23.840813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fac960-61a7-4838-bc18-eed2b7b60046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:23.841486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fac960-61a7-4838-bc18-eed2b7b60046 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.841777 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fac960-61a7-4838-bc18-eed2b7b60046 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.842066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fac960-61a7-4838-bc18-eed2b7b60046 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.842830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fac960-61a7-4838-bc18-eed2b7b60046 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:23.843141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fac960-61a7-4838-bc18-eed2b7b60046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:23.844518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fac960-61a7-4838-bc18-eed2b7b60046 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:23.852662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 38/76] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:23.853934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f09e4-0445-43a9-9ee4-06288685b618 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.854346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f09e4-0445-43a9-9ee4-06288685b618 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.854710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f09e4-0445-43a9-9ee4-06288685b618 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.858498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 39/77] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:23.858498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f09e4-0445-43a9-9ee4-06288685b618 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:23.917326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1e538-cde4-401e-9160-34ef5f5e459a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:23.952330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7f06b-d493-4e7d-b23a-da5f2dfea20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:23.953322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7f06b-d493-4e7d-b23a-da5f2dfea20b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.953754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7f06b-d493-4e7d-b23a-da5f2dfea20b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.954236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7f06b-d493-4e7d-b23a-da5f2dfea20b None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:23.955383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7f06b-d493-4e7d-b23a-da5f2dfea20b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:23.955939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7f06b-d493-4e7d-b23a-da5f2dfea20b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:23.957304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f09e4-0445-43a9-9ee4-06288685b618 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:23.957929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7f06b-d493-4e7d-b23a-da5f2dfea20b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:23.967161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 39/78] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:51:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:23.967975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7f06b-d493-4e7d-b23a-da5f2dfea20b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:23.992061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00996f30-3abb-4a7f-bdb8-8d4b93e62231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:23.992859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00996f30-3abb-4a7f-bdb8-8d4b93e62231 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:23.993145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00996f30-3abb-4a7f-bdb8-8d4b93e62231 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:23.993424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00996f30-3abb-4a7f-bdb8-8d4b93e62231 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.004242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00996f30-3abb-4a7f-bdb8-8d4b93e62231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:24.007802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-00996f30-3abb-4a7f-bdb8-8d4b93e62231 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:24.011974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:24.012899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:24.013836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 40/79] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:24.014822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00996f30-3abb-4a7f-bdb8-8d4b93e62231 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:24.066491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:24.079489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7f06b-d493-4e7d-b23a-da5f2dfea20b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:24.112150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045162bb-78b3-4a96-8f74-dbbb28dcdbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:24.113009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045162bb-78b3-4a96-8f74-dbbb28dcdbdf None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.113213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00996f30-3abb-4a7f-bdb8-8d4b93e62231 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:24.113306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045162bb-78b3-4a96-8f74-dbbb28dcdbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.113571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045162bb-78b3-4a96-8f74-dbbb28dcdbdf None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.114799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045162bb-78b3-4a96-8f74-dbbb28dcdbdf None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:24.116239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045162bb-78b3-4a96-8f74-dbbb28dcdbdf None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:24.117979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 40/80] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:23 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:24.119224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39fcb0-5742-46e1-a94f-d7080aa668cb None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.119520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39fcb0-5742-46e1-a94f-d7080aa668cb None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.119807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc39fcb0-5742-46e1-a94f-d7080aa668cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.136706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dc39fcb0-5742-46e1-a94f-d7080aa668cb None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:24.137944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dc39fcb0-5742-46e1-a94f-d7080aa668cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:24.147792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509d93f-0621-4878-9268-22065aa2cf09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:24.148656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509d93f-0621-4878-9268-22065aa2cf09 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.148937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509d93f-0621-4878-9268-22065aa2cf09 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.149212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509d93f-0621-4878-9268-22065aa2cf09 None admin] PATH_INFO: `/v3/users/a05c37bdf694480a8a14de0b18b2da4b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.150423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509d93f-0621-4878-9268-22065aa2cf09 None admin] RBAC: Authorizing `identity:get_user(user_id=a05c37bdf694480a8a14de0b18b2da4b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:24.151843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509d93f-0621-4878-9268-22065aa2cf09 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:24.154196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 41/81] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:24 2022] GET /identity/v3/users/a05c37bdf694480a8a14de0b18b2da4b => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:24.154770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509d93f-0621-4878-9268-22065aa2cf09 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:24.204237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 41/82] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:24.205150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39fcb0-5742-46e1-a94f-d7080aa668cb None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:24.232827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:24.266680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509d93f-0621-4878-9268-22065aa2cf09 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:24.286193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:24.301220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f6f3c-a819-4b07-86b3-673bc1653b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:24.302124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6f3c-a819-4b07-86b3-673bc1653b9a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.302408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6f3c-a819-4b07-86b3-673bc1653b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.302683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f6f3c-a819-4b07-86b3-673bc1653b9a None admin] PATH_INFO: `/v3/users/19c972280f644c03b032e16fcb039658` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.303521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc39fcb0-5742-46e1-a94f-d7080aa668cb None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:24.303861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f6f3c-a819-4b07-86b3-673bc1653b9a None admin] RBAC: Authorizing `identity:get_user(user_id=19c972280f644c03b032e16fcb039658)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:24.305324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f6f3c-a819-4b07-86b3-673bc1653b9a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:24.307010 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 42/83] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:24 2022] GET /identity/v3/users/19c972280f644c03b032e16fcb039658 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:24.307765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f6f3c-a819-4b07-86b3-673bc1653b9a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:24.336530 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29537e94-de2a-434b-8d99-c3333d20030d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:24.337136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29537e94-de2a-434b-8d99-c3333d20030d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.337422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29537e94-de2a-434b-8d99-c3333d20030d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.337700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29537e94-de2a-434b-8d99-c3333d20030d None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.338500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29537e94-de2a-434b-8d99-c3333d20030d None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:24.338810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29537e94-de2a-434b-8d99-c3333d20030d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:24.340746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29537e94-de2a-434b-8d99-c3333d20030d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:24.349727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 42/84] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:24.350865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29537e94-de2a-434b-8d99-c3333d20030d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:24.420294 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f6f3c-a819-4b07-86b3-673bc1653b9a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:24.439811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:24.449456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29537e94-de2a-434b-8d99-c3333d20030d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:24.452558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0c512-93e0-480f-9f53-c070fc82f9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:24.453217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c512-93e0-480f-9f53-c070fc82f9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.453508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c512-93e0-480f-9f53-c070fc82f9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.453853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0c512-93e0-480f-9f53-c070fc82f9a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.454726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0c512-93e0-480f-9f53-c070fc82f9a2 None admin] RBAC: Adding query filter params (role.id=e1ed407e2d924fac8012250bb2ecf94f, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c, user.id=19c972280f644c03b032e16fcb039658) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:24.455051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0c512-93e0-480f-9f53-c070fc82f9a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:24.456504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e0c512-93e0-480f-9f53-c070fc82f9a2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:24.461991 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 43/85] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:24 2022] GET /identity/v3/role_assignments?user.id=19c972280f644c03b032e16fcb039658&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:24.462794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0c512-93e0-480f-9f53-c070fc82f9a2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:24.487300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa9f33-0eff-48df-ace0-64fa6e1c96b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:24.488228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9f33-0eff-48df-ace0-64fa6e1c96b5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.488660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9f33-0eff-48df-ace0-64fa6e1c96b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.489083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aa9f33-0eff-48df-ace0-64fa6e1c96b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.519343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:24.569791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa9f33-0eff-48df-ace0-64fa6e1c96b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:24.570841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0c512-93e0-480f-9f53-c070fc82f9a2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:24.571761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aa9f33-0eff-48df-ace0-64fa6e1c96b5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:24.608808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17a0e5-f2cc-400d-9efe-fe1abb9e7af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:24.609538 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17a0e5-f2cc-400d-9efe-fe1abb9e7af6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.609828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17a0e5-f2cc-400d-9efe-fe1abb9e7af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.610132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17a0e5-f2cc-400d-9efe-fe1abb9e7af6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.611696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17a0e5-f2cc-400d-9efe-fe1abb9e7af6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:24.612795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17a0e5-f2cc-400d-9efe-fe1abb9e7af6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:24.614498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 44/86] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:24.615370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17a0e5-f2cc-400d-9efe-fe1abb9e7af6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:24.669557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:24.680929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 43/87] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:24.681935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa9f33-0eff-48df-ace0-64fa6e1c96b5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:24.731927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17a0e5-f2cc-400d-9efe-fe1abb9e7af6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:24.754587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:24.769439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701f2e2-ccbd-4880-9bd5-c469c12669d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:24.770114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701f2e2-ccbd-4880-9bd5-c469c12669d7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.770400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701f2e2-ccbd-4880-9bd5-c469c12669d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.770673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8701f2e2-ccbd-4880-9bd5-c469c12669d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.787422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aa9f33-0eff-48df-ace0-64fa6e1c96b5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:24.825171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64967-c18b-4f0b-90f5-35e9d9b471f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:24.826346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64967-c18b-4f0b-90f5-35e9d9b471f9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.826773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64967-c18b-4f0b-90f5-35e9d9b471f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.827219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b64967-c18b-4f0b-90f5-35e9d9b471f9 None admin] PATH_INFO: `/v3/users/19c972280f644c03b032e16fcb039658` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.828689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64967-c18b-4f0b-90f5-35e9d9b471f9 None admin] RBAC: Authorizing `identity:get_user(user_id=19c972280f644c03b032e16fcb039658)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:24.830388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b64967-c18b-4f0b-90f5-35e9d9b471f9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:24.832302 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 44/88] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:24 2022] GET /identity/v3/users/19c972280f644c03b032e16fcb039658 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:24.833171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64967-c18b-4f0b-90f5-35e9d9b471f9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:24.839913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701f2e2-ccbd-4880-9bd5-c469c12669d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:24.842059 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8701f2e2-ccbd-4880-9bd5-c469c12669d7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:24.906128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:24.949337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b64967-c18b-4f0b-90f5-35e9d9b471f9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:24.957296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 45/89] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:24.958433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701f2e2-ccbd-4880-9bd5-c469c12669d7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:24.979651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:24.986974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4acee-b798-40ab-a7d5-342f214441ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:24.987658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4acee-b798-40ab-a7d5-342f214441ee None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:24.987947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4acee-b798-40ab-a7d5-342f214441ee None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:24.988229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be4acee-b798-40ab-a7d5-342f214441ee None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:24.988951 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4acee-b798-40ab-a7d5-342f214441ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:24.990415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be4acee-b798-40ab-a7d5-342f214441ee None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:25.026081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 45/90] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:51:24 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:51:25.026081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4acee-b798-40ab-a7d5-342f214441ee None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:25.087734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8701f2e2-ccbd-4880-9bd5-c469c12669d7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:25.126733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da8416-4531-4a9c-9d2f-4e230e8e0da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:25.127794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8416-4531-4a9c-9d2f-4e230e8e0da2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:25.128184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8416-4531-4a9c-9d2f-4e230e8e0da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:25.128565 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da8416-4531-4a9c-9d2f-4e230e8e0da2 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:25.129946 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da8416-4531-4a9c-9d2f-4e230e8e0da2 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:25.131551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43da8416-4531-4a9c-9d2f-4e230e8e0da2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:25.133823 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 46/91] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:24 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:25.134861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da8416-4531-4a9c-9d2f-4e230e8e0da2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:25.177843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be4acee-b798-40ab-a7d5-342f214441ee None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:25.196451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:25.223909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad35b8-ed44-4c42-9ad7-fbf50211b942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:25.224574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad35b8-ed44-4c42-9ad7-fbf50211b942 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:25.224862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad35b8-ed44-4c42-9ad7-fbf50211b942 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:25.225145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad35b8-ed44-4c42-9ad7-fbf50211b942 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:25.241513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:25.261794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da8416-4531-4a9c-9d2f-4e230e8e0da2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:25.308606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11502ca-fa50-4960-8860-147ec71da2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:25.310033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11502ca-fa50-4960-8860-147ec71da2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:25.310332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11502ca-fa50-4960-8860-147ec71da2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:25.310609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11502ca-fa50-4960-8860-147ec71da2e0 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:25.311855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11502ca-fa50-4960-8860-147ec71da2e0 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:25.313295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11502ca-fa50-4960-8860-147ec71da2e0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:25.315153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 47/92] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:25 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:25.315943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11502ca-fa50-4960-8860-147ec71da2e0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:25.316907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad35b8-ed44-4c42-9ad7-fbf50211b942 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:25.318642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad35b8-ed44-4c42-9ad7-fbf50211b942 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:25.453200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 46/93] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:25.454684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad35b8-ed44-4c42-9ad7-fbf50211b942 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:25.456666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11502ca-fa50-4960-8860-147ec71da2e0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:25.474672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:25.513414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0867d-e54b-4574-9b66-c85c920dd2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:25.514124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0867d-e54b-4574-9b66-c85c920dd2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:25.514419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0867d-e54b-4574-9b66-c85c920dd2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:25.514702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d0867d-e54b-4574-9b66-c85c920dd2b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:25.515977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0867d-e54b-4574-9b66-c85c920dd2b9 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c, user.id=a05c37bdf694480a8a14de0b18b2da4b) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:25.516302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0867d-e54b-4574-9b66-c85c920dd2b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:25.518111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d0867d-e54b-4574-9b66-c85c920dd2b9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:25.533737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 48/94] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:25 2022] GET /identity/v3/role_assignments?user.id=a05c37bdf694480a8a14de0b18b2da4b&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:25.536026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0867d-e54b-4574-9b66-c85c920dd2b9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:25.623926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:25.676791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad35b8-ed44-4c42-9ad7-fbf50211b942 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:25.750698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea6175-12e9-4d37-a475-817bf9c6b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:25.751687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea6175-12e9-4d37-a475-817bf9c6b95d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:25.752178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea6175-12e9-4d37-a475-817bf9c6b95d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:25.752663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea6175-12e9-4d37-a475-817bf9c6b95d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:25.753691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea6175-12e9-4d37-a475-817bf9c6b95d None admin] RBAC: Adding query filter params (role.id=4f31bdaad0194bbd8fab39617059d14a, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c, user.id=19c972280f644c03b032e16fcb039658) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:25.754334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea6175-12e9-4d37-a475-817bf9c6b95d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:25.755976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea6175-12e9-4d37-a475-817bf9c6b95d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:25.769228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 47/95] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:25 2022] GET /identity/v3/role_assignments?user.id=19c972280f644c03b032e16fcb039658&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=4f31bdaad0194bbd8fab39617059d14a => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:25.792883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d0867d-e54b-4574-9b66-c85c920dd2b9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:25.846462 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14435621-8f03-495e-90c8-deedd0580067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:25.847377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435621-8f03-495e-90c8-deedd0580067 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:25.847668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435621-8f03-495e-90c8-deedd0580067 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:25.847951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14435621-8f03-495e-90c8-deedd0580067 None admin] PATH_INFO: `/v3/projects/fd662c840e364e918c1973e41dd5aa73` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:25.855367 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14435621-8f03-495e-90c8-deedd0580067 None admin] RBAC: Authorizing `identity:get_project(project_id=fd662c840e364e918c1973e41dd5aa73)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:25.856505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14435621-8f03-495e-90c8-deedd0580067 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:25.858456 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 49/96] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:25 2022] GET /identity/v3/projects/fd662c840e364e918c1973e41dd5aa73 => generated 312 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:25.867100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea6175-12e9-4d37-a475-817bf9c6b95d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:26.004143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:26.028478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea6175-12e9-4d37-a475-817bf9c6b95d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:26.065111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b88018e-2fd6-4b15-b926-f7decf67c82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:26.065770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88018e-2fd6-4b15-b926-f7decf67c82e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:26.066069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88018e-2fd6-4b15-b926-f7decf67c82e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:26.066369 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b88018e-2fd6-4b15-b926-f7decf67c82e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:26.067214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88018e-2fd6-4b15-b926-f7decf67c82e None admin] RBAC: Adding query filter params (role.id=e1ed407e2d924fac8012250bb2ecf94f, scope.project.id=fd662c840e364e918c1973e41dd5aa73, user.id=19c972280f644c03b032e16fcb039658) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:26.067528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88018e-2fd6-4b15-b926-f7decf67c82e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:26.068968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b88018e-2fd6-4b15-b926-f7decf67c82e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:26.075448 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 48/97] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:25 2022] GET /identity/v3/role_assignments?user.id=19c972280f644c03b032e16fcb039658&scope.project.id=fd662c840e364e918c1973e41dd5aa73&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:27.318602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d33515-b46a-4707-813d-399e9dd37ebf None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:27.319108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d33515-b46a-4707-813d-399e9dd37ebf None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:27.319181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d33515-b46a-4707-813d-399e9dd37ebf None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:27.320839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 50/98] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:27.335975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb15502-53da-4a0b-acf7-9c3171957d35 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:27.336425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb15502-53da-4a0b-acf7-9c3171957d35 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:27.336604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb15502-53da-4a0b-acf7-9c3171957d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:27.359175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bcb15502-53da-4a0b-acf7-9c3171957d35 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:27.360381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bcb15502-53da-4a0b-acf7-9c3171957d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:27.445771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 49/99] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:27.487516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d33515-b46a-4707-813d-399e9dd37ebf None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:27.628853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:27.650717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d33515-b46a-4707-813d-399e9dd37ebf None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:27.700669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628bac-499a-45c0-b724-1180f55e2269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:27.701477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628bac-499a-45c0-b724-1180f55e2269 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:27.701763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628bac-499a-45c0-b724-1180f55e2269 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:27.702079 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3628bac-499a-45c0-b724-1180f55e2269 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:27.715917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116661d-6613-4f0d-9085-08fa1401e04f None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:27.716213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116661d-6613-4f0d-9085-08fa1401e04f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:27.716493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116661d-6613-4f0d-9085-08fa1401e04f None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:27.718182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 50/100] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:27.718659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3628bac-499a-45c0-b724-1180f55e2269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:27.723414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe05236-944d-4bde-88c0-e77c6edc65b2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:27.723703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe05236-944d-4bde-88c0-e77c6edc65b2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:27.723981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe05236-944d-4bde-88c0-e77c6edc65b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e3628bac-499a-45c0-b724-1180f55e2269 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:27.726997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:27.727820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:27.728578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 51/101] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:27.734587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628bac-499a-45c0-b724-1180f55e2269 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:27.759742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bbe05236-944d-4bde-88c0-e77c6edc65b2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:27.760979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bbe05236-944d-4bde-88c0-e77c6edc65b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:27.880469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 51/102] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:27.927622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe05236-944d-4bde-88c0-e77c6edc65b2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:27.930118 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:27.934016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3628bac-499a-45c0-b724-1180f55e2269 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:28.007611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43820563-4545-4540-9fce-1c355c63d4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:28.008291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43820563-4545-4540-9fce-1c355c63d4aa None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.008577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43820563-4545-4540-9fce-1c355c63d4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.008857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43820563-4545-4540-9fce-1c355c63d4aa None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.009598 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43820563-4545-4540-9fce-1c355c63d4aa None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:28.010106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43820563-4545-4540-9fce-1c355c63d4aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:28.011620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43820563-4545-4540-9fce-1c355c63d4aa None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:28.029997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 52/103] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:28.035166 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43820563-4545-4540-9fce-1c355c63d4aa None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:28.044865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe05236-944d-4bde-88c0-e77c6edc65b2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:28.057651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:28.085571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8be32-1c99-49ca-8044-a5c9c4e5482b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:28.086394 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8be32-1c99-49ca-8044-a5c9c4e5482b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.086730 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8be32-1c99-49ca-8044-a5c9c4e5482b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.087010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d8be32-1c99-49ca-8044-a5c9c4e5482b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.119041 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d8be32-1c99-49ca-8044-a5c9c4e5482b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-28d8be32-1c99-49ca-8044-a5c9c4e5482b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:28.127249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:28.128078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:28.128806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:28.134596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 52/104] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:28.139850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8be32-1c99-49ca-8044-a5c9c4e5482b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:28.256039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43820563-4545-4540-9fce-1c355c63d4aa None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:28.295826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:28.327632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bf99a-81b1-43e3-a653-05ae168c4a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:28.328519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bf99a-81b1-43e3-a653-05ae168c4a95 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.328800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bf99a-81b1-43e3-a653-05ae168c4a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.329073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420bf99a-81b1-43e3-a653-05ae168c4a95 None admin] PATH_INFO: `/v3/users/19c972280f644c03b032e16fcb039658` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.332908 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bf99a-81b1-43e3-a653-05ae168c4a95 None admin] RBAC: Authorizing `identity:get_user(user_id=19c972280f644c03b032e16fcb039658)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:28.334893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420bf99a-81b1-43e3-a653-05ae168c4a95 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:28.336589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 53/105] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:28 2022] GET /identity/v3/users/19c972280f644c03b032e16fcb039658 => generated 275 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:28.342723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bf99a-81b1-43e3-a653-05ae168c4a95 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:28.387778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d8be32-1c99-49ca-8044-a5c9c4e5482b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:28.457764 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:28.481741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3906dd-7511-4bcc-9de1-65ac46fae1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:28.482453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3906dd-7511-4bcc-9de1-65ac46fae1d4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.482745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3906dd-7511-4bcc-9de1-65ac46fae1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.483028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3906dd-7511-4bcc-9de1-65ac46fae1d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.483793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3906dd-7511-4bcc-9de1-65ac46fae1d4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:28.484107 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3906dd-7511-4bcc-9de1-65ac46fae1d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:28.485630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3906dd-7511-4bcc-9de1-65ac46fae1d4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:28.500308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420bf99a-81b1-43e3-a653-05ae168c4a95 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:28.503936 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 53/106] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:28.505265 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f8cbe-d3d7-4333-b3cf-bed7d58e2ca7 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.505562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f8cbe-d3d7-4333-b3cf-bed7d58e2ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.505846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f8cbe-d3d7-4333-b3cf-bed7d58e2ca7 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.516570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 54/107] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:28 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:28.520556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f8cbe-d3d7-4333-b3cf-bed7d58e2ca7 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:28.557603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc9735-6d58-4e99-b983-cbeafc4cf5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:28.559440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc9735-6d58-4e99-b983-cbeafc4cf5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.559922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc9735-6d58-4e99-b983-cbeafc4cf5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.560391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc9735-6d58-4e99-b983-cbeafc4cf5a9 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.562778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc9735-6d58-4e99-b983-cbeafc4cf5a9 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:28.564244 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc9735-6d58-4e99-b983-cbeafc4cf5a9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:28.570091 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 54/108] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:28 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:28.570091 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ef414-bc8c-4df4-893a-a8794451934c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.570091 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ef414-bc8c-4df4-893a-a8794451934c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.570091 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ef414-bc8c-4df4-893a-a8794451934c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.593526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4c5ef414-bc8c-4df4-893a-a8794451934c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:28.595574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4c5ef414-bc8c-4df4-893a-a8794451934c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:28.648232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f8cbe-d3d7-4333-b3cf-bed7d58e2ca7 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:28.679531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 55/109] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:28.680574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:28.680754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4c431-0d97-4f5b-8b5a-730f8afb81ec None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.680997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4c431-0d97-4f5b-8b5a-730f8afb81ec None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.681284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4c431-0d97-4f5b-8b5a-730f8afb81ec None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.683112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 56/110] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:28.687643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4c431-0d97-4f5b-8b5a-730f8afb81ec None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:28.701314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0194f-fe77-4b54-9811-405078f3531f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:28.703278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0194f-fe77-4b54-9811-405078f3531f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.704068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0194f-fe77-4b54-9811-405078f3531f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.704629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0194f-fe77-4b54-9811-405078f3531f None admin] PATH_INFO: `/v3/users/581bd67ace7c43e5a7d3bb35a0182692` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.724560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0194f-fe77-4b54-9811-405078f3531f None admin] RBAC: Authorizing `identity:get_user(user_id=581bd67ace7c43e5a7d3bb35a0182692)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:28.727188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0194f-fe77-4b54-9811-405078f3531f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:28.731234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 55/111] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:28 2022] GET /identity/v3/users/581bd67ace7c43e5a7d3bb35a0182692 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:28.732893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3211b02-15a8-4c45-8dce-3ba04395dc59 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.733386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3211b02-15a8-4c45-8dce-3ba04395dc59 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.733928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3211b02-15a8-4c45-8dce-3ba04395dc59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.754740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e3211b02-15a8-4c45-8dce-3ba04395dc59 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:28.757035 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e3211b02-15a8-4c45-8dce-3ba04395dc59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:28.793086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:28.822894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4c431-0d97-4f5b-8b5a-730f8afb81ec None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:28.855359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 56/112] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:28.858193 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3211b02-15a8-4c45-8dce-3ba04395dc59 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:28.876117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2d2f5-a2ce-4c3e-92e8-baceaa77d0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:28.877036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2d2f5-a2ce-4c3e-92e8-baceaa77d0d0 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:28.877297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2d2f5-a2ce-4c3e-92e8-baceaa77d0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:28.877621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2d2f5-a2ce-4c3e-92e8-baceaa77d0d0 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/users/19c972280f644c03b032e16fcb039658/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:28.879638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2d2f5-a2ce-4c3e-92e8-baceaa77d0d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e855e8bdc61c4a8a91cadb04fe89906c, user_id=19c972280f644c03b032e16fcb039658, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:28.881094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b2d2f5-a2ce-4c3e-92e8-baceaa77d0d0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:28.895598 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 57/113] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:51:28 2022] PUT /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/users/19c972280f644c03b032e16fcb039658/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:51:28.896209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2d2f5-a2ce-4c3e-92e8-baceaa77d0d0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:28.979379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3211b02-15a8-4c45-8dce-3ba04395dc59 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:28.995390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:29.013023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2d2f5-a2ce-4c3e-92e8-baceaa77d0d0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:29.021295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2491774-f925-4142-8e48-caeb5b211d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:29.022166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2491774-f925-4142-8e48-caeb5b211d3a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.022457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2491774-f925-4142-8e48-caeb5b211d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.023166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2491774-f925-4142-8e48-caeb5b211d3a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.024794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2491774-f925-4142-8e48-caeb5b211d3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:29.026300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2491774-f925-4142-8e48-caeb5b211d3a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:29.028563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 57/114] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:29.029845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132d844-c649-45c7-b92c-77e6cd480dbb None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.030255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132d844-c649-45c7-b92c-77e6cd480dbb None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.030440 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3132d844-c649-45c7-b92c-77e6cd480dbb None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.031782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 58/115] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:29.035954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4646-ae55-4c1f-ac3e-995dc81dca93 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.036639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4646-ae55-4c1f-ac3e-995dc81dca93 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.036933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7e4646-ae55-4c1f-ac3e-995dc81dca93 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.038701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 59/116] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:29 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:29.039332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e4646-ae55-4c1f-ac3e-995dc81dca93 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:29.070750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5912fd64-5a01-42f8-8f1f-57f03f3f57e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:29.071444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912fd64-5a01-42f8-8f1f-57f03f3f57e9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.071735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912fd64-5a01-42f8-8f1f-57f03f3f57e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.072018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5912fd64-5a01-42f8-8f1f-57f03f3f57e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.076816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:29.146744 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912fd64-5a01-42f8-8f1f-57f03f3f57e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:29.148216 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5912fd64-5a01-42f8-8f1f-57f03f3f57e9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:29.156593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7e4646-ae55-4c1f-ac3e-995dc81dca93 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:29.193676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d8d1d-393c-4768-8d60-d680382d834d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:29.194507 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d8d1d-393c-4768-8d60-d680382d834d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.194802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d8d1d-393c-4768-8d60-d680382d834d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.195088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d8d1d-393c-4768-8d60-d680382d834d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.206559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d8d1d-393c-4768-8d60-d680382d834d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-c25d8d1d-393c-4768-8d60-d680382d834d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:29.214248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:29.215668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 60/117] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:51:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:29.217209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d8d1d-393c-4768-8d60-d680382d834d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:29.260160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 58/118] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:29.260160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca86f6-a2ed-49ec-b138-6a5e67b2badf None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.260160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca86f6-a2ed-49ec-b138-6a5e67b2badf None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.260160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ca86f6-a2ed-49ec-b138-6a5e67b2badf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.265572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:29.274803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-94ca86f6-a2ed-49ec-b138-6a5e67b2badf None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:29.275990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-94ca86f6-a2ed-49ec-b138-6a5e67b2badf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:29.308317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:29.330348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d8d1d-393c-4768-8d60-d680382d834d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:29.354917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 59/119] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:29.354917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6b05f-b270-4eae-a531-088cb1d45a2d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.354917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6b05f-b270-4eae-a531-088cb1d45a2d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.354917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b6b05f-b270-4eae-a531-088cb1d45a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.374074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c3b6b05f-b270-4eae-a531-088cb1d45a2d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:29.375690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c3b6b05f-b270-4eae-a531-088cb1d45a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:29.384566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5212a63-a048-4b51-a9be-e97eaf61c5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:29.385272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5212a63-a048-4b51-a9be-e97eaf61c5ff None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.385538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5212a63-a048-4b51-a9be-e97eaf61c5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.385821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5212a63-a048-4b51-a9be-e97eaf61c5ff None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.386576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5212a63-a048-4b51-a9be-e97eaf61c5ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:29.388560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5212a63-a048-4b51-a9be-e97eaf61c5ff None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:29.416101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 61/120] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:51:29 2022] POST /identity/v3/projects => generated 302 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:51:29.416775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5212a63-a048-4b51-a9be-e97eaf61c5ff None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:29.469015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 60/121] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:29.470505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6b05f-b270-4eae-a531-088cb1d45a2d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:29.546583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:29.559658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5212a63-a048-4b51-a9be-e97eaf61c5ff None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:29.603490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cd33b-638b-4855-9f8c-5e2e1c5f9d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:29.604549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cd33b-638b-4855-9f8c-5e2e1c5f9d43 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.604842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cd33b-638b-4855-9f8c-5e2e1c5f9d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.605235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248cd33b-638b-4855-9f8c-5e2e1c5f9d43 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.605999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cd33b-638b-4855-9f8c-5e2e1c5f9d43 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:29.606384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cd33b-638b-4855-9f8c-5e2e1c5f9d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:29.607785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248cd33b-638b-4855-9f8c-5e2e1c5f9d43 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:29.607972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:29.617311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 62/122] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:29.618143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b6b05f-b270-4eae-a531-088cb1d45a2d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:29.618320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cd33b-638b-4855-9f8c-5e2e1c5f9d43 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:29.663355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4954a-046a-43e9-b4f9-2c37c7d41860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:29.664238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4954a-046a-43e9-b4f9-2c37c7d41860 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.664528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4954a-046a-43e9-b4f9-2c37c7d41860 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.664812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e4954a-046a-43e9-b4f9-2c37c7d41860 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.666086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4954a-046a-43e9-b4f9-2c37c7d41860 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:29.667587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e4954a-046a-43e9-b4f9-2c37c7d41860 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:29.670113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 61/123] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:29 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:29.671363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4954a-046a-43e9-b4f9-2c37c7d41860 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:29.741532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248cd33b-638b-4855-9f8c-5e2e1c5f9d43 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:29.785273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1245bfb-32b9-4867-900c-d55fc7ea5e03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:29.787095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1245bfb-32b9-4867-900c-d55fc7ea5e03 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.787859 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1245bfb-32b9-4867-900c-d55fc7ea5e03 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.788338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1245bfb-32b9-4867-900c-d55fc7ea5e03 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.800767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e4954a-046a-43e9-b4f9-2c37c7d41860 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:29.802993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1245bfb-32b9-4867-900c-d55fc7ea5e03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f1245bfb-32b9-4867-900c-d55fc7ea5e03 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:29.813748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:29.814607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:29.815339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:29.823156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 63/124] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:29.824293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1245bfb-32b9-4867-900c-d55fc7ea5e03 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:29.840854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:29.844714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827dc12e-108f-4bd8-89b0-bf6112aab1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:29.845851 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827dc12e-108f-4bd8-89b0-bf6112aab1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.846129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827dc12e-108f-4bd8-89b0-bf6112aab1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.846465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827dc12e-108f-4bd8-89b0-bf6112aab1a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.862857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827dc12e-108f-4bd8-89b0-bf6112aab1a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-827dc12e-108f-4bd8-89b0-bf6112aab1a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:29.877504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:29.878539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:29.879271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 62/125] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:51:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:29.879888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827dc12e-108f-4bd8-89b0-bf6112aab1a9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:29.911830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:29.938446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1245bfb-32b9-4867-900c-d55fc7ea5e03 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:29.972988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738c607-765e-4e08-b369-84537ba8b7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:29.974246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c607-765e-4e08-b369-84537ba8b7ad None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:29.974732 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c607-765e-4e08-b369-84537ba8b7ad None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:29.975441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738c607-765e-4e08-b369-84537ba8b7ad None admin] PATH_INFO: `/v3/users/a05c37bdf694480a8a14de0b18b2da4b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:29.977033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738c607-765e-4e08-b369-84537ba8b7ad None admin] RBAC: Authorizing `identity:get_user(user_id=a05c37bdf694480a8a14de0b18b2da4b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:29.978677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738c607-765e-4e08-b369-84537ba8b7ad None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:29.981511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 64/126] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:29 2022] GET /identity/v3/users/a05c37bdf694480a8a14de0b18b2da4b => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:29.982534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738c607-765e-4e08-b369-84537ba8b7ad None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:29.993889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827dc12e-108f-4bd8-89b0-bf6112aab1a9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:30.045787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4156eb-682b-4584-850e-7bfa867063c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:30.046864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4156eb-682b-4584-850e-7bfa867063c3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:30.047633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4156eb-682b-4584-850e-7bfa867063c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:30.048084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4156eb-682b-4584-850e-7bfa867063c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:30.049753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4156eb-682b-4584-850e-7bfa867063c3 None admin] RBAC: Adding query filter params (role.id=c2c12c04e72a4116bf5d8e58c2cedc7b, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c, user.id=581bd67ace7c43e5a7d3bb35a0182692) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:30.050267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4156eb-682b-4584-850e-7bfa867063c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:30.052980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4156eb-682b-4584-850e-7bfa867063c3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:30.061820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 63/127] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:29 2022] GET /identity/v3/role_assignments?user.id=581bd67ace7c43e5a7d3bb35a0182692&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=c2c12c04e72a4116bf5d8e58c2cedc7b => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:30.062856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4156eb-682b-4584-850e-7bfa867063c3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:30.095298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:30.159928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738c607-765e-4e08-b369-84537ba8b7ad None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:30.174428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:30.186861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4156eb-682b-4584-850e-7bfa867063c3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:30.226460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136ebb9-c5dd-455a-bb8d-0fec5012a968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:30.227579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136ebb9-c5dd-455a-bb8d-0fec5012a968 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:30.228168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136ebb9-c5dd-455a-bb8d-0fec5012a968 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:30.228618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136ebb9-c5dd-455a-bb8d-0fec5012a968 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:30.229545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136ebb9-c5dd-455a-bb8d-0fec5012a968 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:30.230337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136ebb9-c5dd-455a-bb8d-0fec5012a968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:30.233105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136ebb9-c5dd-455a-bb8d-0fec5012a968 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:30.243342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 64/128] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:51:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:30.244684 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136ebb9-c5dd-455a-bb8d-0fec5012a968 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:30.256188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0ee6d-14c5-4652-ab52-7c4c935f12a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:30.256940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0ee6d-14c5-4652-ab52-7c4c935f12a0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:30.257236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0ee6d-14c5-4652-ab52-7c4c935f12a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:30.257521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e0ee6d-14c5-4652-ab52-7c4c935f12a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:30.258359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0ee6d-14c5-4652-ab52-7c4c935f12a0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:30.258622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0ee6d-14c5-4652-ab52-7c4c935f12a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:30.260162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e0ee6d-14c5-4652-ab52-7c4c935f12a0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:30.288568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 65/129] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:30.289456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0ee6d-14c5-4652-ab52-7c4c935f12a0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:30.381110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136ebb9-c5dd-455a-bb8d-0fec5012a968 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:30.427200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e0ee6d-14c5-4652-ab52-7c4c935f12a0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:30.460057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a503d-47a1-4ac5-82b9-0536d51adad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:30.461374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a503d-47a1-4ac5-82b9-0536d51adad9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:30.461966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a503d-47a1-4ac5-82b9-0536d51adad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:30.462533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a503d-47a1-4ac5-82b9-0536d51adad9 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:30.464246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a503d-47a1-4ac5-82b9-0536d51adad9 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:30.464652 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b992b-7df3-4568-933e-cb52b8c2a0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:30.466106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b992b-7df3-4568-933e-cb52b8c2a0be None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:30.466401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b992b-7df3-4568-933e-cb52b8c2a0be None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:30.466682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619b992b-7df3-4568-933e-cb52b8c2a0be None admin] PATH_INFO: `/v3/users/19c972280f644c03b032e16fcb039658` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:30.467442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531a503d-47a1-4ac5-82b9-0536d51adad9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:30.467903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b992b-7df3-4568-933e-cb52b8c2a0be None admin] RBAC: Authorizing `identity:get_user(user_id=19c972280f644c03b032e16fcb039658)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:30.469355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619b992b-7df3-4568-933e-cb52b8c2a0be None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:30.469843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 65/130] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:30 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:30.470803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a503d-47a1-4ac5-82b9-0536d51adad9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:30.471232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 66/131] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:30 2022] GET /identity/v3/users/19c972280f644c03b032e16fcb039658 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:30.479841 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b992b-7df3-4568-933e-cb52b8c2a0be None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:30.513148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:30.513761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:30.611555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619b992b-7df3-4568-933e-cb52b8c2a0be None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:30.649466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fc2a7-6e66-4423-a1df-22b137737f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:30.650347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fc2a7-6e66-4423-a1df-22b137737f3f None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:30.650644 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fc2a7-6e66-4423-a1df-22b137737f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:30.651376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fc2a7-6e66-4423-a1df-22b137737f3f None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/users/a05c37bdf694480a8a14de0b18b2da4b/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:30.653953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fc2a7-6e66-4423-a1df-22b137737f3f None admin] RBAC: Authorizing `identity:create_grant(project_id=e855e8bdc61c4a8a91cadb04fe89906c, user_id=a05c37bdf694480a8a14de0b18b2da4b, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:30.655626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fc2a7-6e66-4423-a1df-22b137737f3f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:30.677403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 67/132] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:51:30 2022] PUT /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/users/a05c37bdf694480a8a14de0b18b2da4b/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:51:30.682362 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fc2a7-6e66-4423-a1df-22b137737f3f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:30.741526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a503d-47a1-4ac5-82b9-0536d51adad9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:30.788795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88b9af-b1f9-475d-bb03-3a9f3e3fbcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:30.789852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88b9af-b1f9-475d-bb03-3a9f3e3fbcf3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:30.790178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88b9af-b1f9-475d-bb03-3a9f3e3fbcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:30.790465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e88b9af-b1f9-475d-bb03-3a9f3e3fbcf3 None admin] PATH_INFO: `/v3/users/19c972280f644c03b032e16fcb039658` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:30.791951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88b9af-b1f9-475d-bb03-3a9f3e3fbcf3 None admin] RBAC: Authorizing `identity:get_user(user_id=19c972280f644c03b032e16fcb039658)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:30.793995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e88b9af-b1f9-475d-bb03-3a9f3e3fbcf3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:30.798849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 66/133] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:30 2022] GET /identity/v3/users/19c972280f644c03b032e16fcb039658 => generated 275 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:30.801767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88b9af-b1f9-475d-bb03-3a9f3e3fbcf3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:30.827606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:30.921962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:30.923940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fc2a7-6e66-4423-a1df-22b137737f3f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:30.939637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e88b9af-b1f9-475d-bb03-3a9f3e3fbcf3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:30.993343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc512e15-b5ba-4b85-83c7-80ba9a45e5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:30.994627 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc512e15-b5ba-4b85-83c7-80ba9a45e5ea None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:30.995177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc512e15-b5ba-4b85-83c7-80ba9a45e5ea None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:30.995649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc512e15-b5ba-4b85-83c7-80ba9a45e5ea None admin] PATH_INFO: `/v3/projects/fd662c840e364e918c1973e41dd5aa73` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:30.997613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc512e15-b5ba-4b85-83c7-80ba9a45e5ea None admin] RBAC: Authorizing `identity:get_project(project_id=fd662c840e364e918c1973e41dd5aa73)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:30.999699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc512e15-b5ba-4b85-83c7-80ba9a45e5ea None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:31.001996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 67/134] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:30 2022] GET /identity/v3/projects/fd662c840e364e918c1973e41dd5aa73 => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:31.007923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc512e15-b5ba-4b85-83c7-80ba9a45e5ea None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:31.022426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5591339-860c-4abc-b96c-b3f9c1433ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:31.023775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5591339-860c-4abc-b96c-b3f9c1433ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:31.024370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5591339-860c-4abc-b96c-b3f9c1433ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:31.026661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5591339-860c-4abc-b96c-b3f9c1433ea2 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:31.028308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5591339-860c-4abc-b96c-b3f9c1433ea2 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:31.030260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5591339-860c-4abc-b96c-b3f9c1433ea2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:31.033978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 68/135] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:30 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:31.041086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5591339-860c-4abc-b96c-b3f9c1433ea2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:31.194311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc512e15-b5ba-4b85-83c7-80ba9a45e5ea None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:31.232240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5591339-860c-4abc-b96c-b3f9c1433ea2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:31.266899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f605e5-28f4-4cf5-9ec1-9d510f171f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:31.267810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f605e5-28f4-4cf5-9ec1-9d510f171f59 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:31.268100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f605e5-28f4-4cf5-9ec1-9d510f171f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:31.268386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f605e5-28f4-4cf5-9ec1-9d510f171f59 None admin] PATH_INFO: `/v3/projects/fd662c840e364e918c1973e41dd5aa73/users/19c972280f644c03b032e16fcb039658/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:31.274346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f605e5-28f4-4cf5-9ec1-9d510f171f59 None admin] RBAC: Authorizing `identity:create_grant(project_id=fd662c840e364e918c1973e41dd5aa73, user_id=19c972280f644c03b032e16fcb039658, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:31.275814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f605e5-28f4-4cf5-9ec1-9d510f171f59 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:31.293604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10454b6e-6c5c-443a-b737-3a8f56489020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:31.294538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10454b6e-6c5c-443a-b737-3a8f56489020 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:31.294828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10454b6e-6c5c-443a-b737-3a8f56489020 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:31.295114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10454b6e-6c5c-443a-b737-3a8f56489020 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/users/19c972280f644c03b032e16fcb039658/roles/4f31bdaad0194bbd8fab39617059d14a` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:31.297527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10454b6e-6c5c-443a-b737-3a8f56489020 None admin] RBAC: Authorizing `identity:create_grant(project_id=e855e8bdc61c4a8a91cadb04fe89906c, user_id=19c972280f644c03b032e16fcb039658, role_id=4f31bdaad0194bbd8fab39617059d14a)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:31.299047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10454b6e-6c5c-443a-b737-3a8f56489020 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:31.309727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 68/136] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:51:31 2022] PUT /identity/v3/projects/fd662c840e364e918c1973e41dd5aa73/users/19c972280f644c03b032e16fcb039658/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 02:51:31.311277 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687b03d-cd2c-4e68-ba6b-b42d1b475735 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:31.311587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687b03d-cd2c-4e68-ba6b-b42d1b475735 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:31.312104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687b03d-cd2c-4e68-ba6b-b42d1b475735 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:31.315262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:31.316190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 69/137] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:31.321561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 69/138] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:51:31 2022] PUT /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/users/19c972280f644c03b032e16fcb039658/roles/4f31bdaad0194bbd8fab39617059d14a => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:51:31.322945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d81bd-bdc7-424d-ab63-a0e28301e755 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:31.323243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d81bd-bdc7-424d-ab63-a0e28301e755 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:31.323529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8d81bd-bdc7-424d-ab63-a0e28301e755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:31.342766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5c8d81bd-bdc7-424d-ab63-a0e28301e755 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:31.344021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5c8d81bd-bdc7-424d-ab63-a0e28301e755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:31.355611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:31.421732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 70/139] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:31.457648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10454b6e-6c5c-443a-b737-3a8f56489020 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:31.608978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10454b6e-6c5c-443a-b737-3a8f56489020 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:31.653147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186a52a-6b7f-45c7-a960-904374f63826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:31.653974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186a52a-6b7f-45c7-a960-904374f63826 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:31.654303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186a52a-6b7f-45c7-a960-904374f63826 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:31.654570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186a52a-6b7f-45c7-a960-904374f63826 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:31.666758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1186a52a-6b7f-45c7-a960-904374f63826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1186a52a-6b7f-45c7-a960-904374f63826 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:31.675303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:31.676172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:31.676909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 70/140] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:31.682727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d81bd-bdc7-424d-ab63-a0e28301e755 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:31.789777 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8d81bd-bdc7-424d-ab63-a0e28301e755 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:31.826005 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60a888c-3387-4120-9ca2-5129a0d8d0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:31.826718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a888c-3387-4120-9ca2-5129a0d8d0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:31.827010 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a888c-3387-4120-9ca2-5129a0d8d0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:31.827292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a888c-3387-4120-9ca2-5129a0d8d0f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:31.828055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a888c-3387-4120-9ca2-5129a0d8d0f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:31.828372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a888c-3387-4120-9ca2-5129a0d8d0f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:31.829766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60a888c-3387-4120-9ca2-5129a0d8d0f2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:31.838444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 71/141] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:31.844146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186a52a-6b7f-45c7-a960-904374f63826 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:31.930842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:31.956046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79711c16-5f92-405b-b3ab-c2b8c271f699 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:31.956351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79711c16-5f92-405b-b3ab-c2b8c271f699 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:31.956622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79711c16-5f92-405b-b3ab-c2b8c271f699 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:31.958060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 72/142] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:51:31.962877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66a29-b90f-420d-9c6c-9624778cf35d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:31.963169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66a29-b90f-420d-9c6c-9624778cf35d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:31.963450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d66a29-b90f-420d-9c6c-9624778cf35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:31.977230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:31.990370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-94d66a29-b90f-420d-9c6c-9624778cf35d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:31.991583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-94d66a29-b90f-420d-9c6c-9624778cf35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:32.016684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186a52a-6b7f-45c7-a960-904374f63826 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:32.074868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927434d9-9649-44ea-bd9f-f23c440a0a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:32.075689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927434d9-9649-44ea-bd9f-f23c440a0a2f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:32.075977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927434d9-9649-44ea-bd9f-f23c440a0a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:32.076262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927434d9-9649-44ea-bd9f-f23c440a0a2f None admin] PATH_INFO: `/v3/users/19c972280f644c03b032e16fcb039658` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:32.077520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927434d9-9649-44ea-bd9f-f23c440a0a2f None admin] RBAC: Authorizing `identity:get_user(user_id=19c972280f644c03b032e16fcb039658)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:32.079001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927434d9-9649-44ea-bd9f-f23c440a0a2f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:32.080849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 71/143] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:31 2022] GET /identity/v3/users/19c972280f644c03b032e16fcb039658 => generated 275 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:32.086119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927434d9-9649-44ea-bd9f-f23c440a0a2f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:32.179976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 73/144] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:32.223133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d66a29-b90f-420d-9c6c-9624778cf35d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:32.236591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927434d9-9649-44ea-bd9f-f23c440a0a2f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:32.283804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:32.298830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03226e87-281d-444e-988b-5864440d4b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:32.299506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03226e87-281d-444e-988b-5864440d4b7f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:32.299805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03226e87-281d-444e-988b-5864440d4b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:32.300108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03226e87-281d-444e-988b-5864440d4b7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:32.366224 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d66a29-b90f-420d-9c6c-9624778cf35d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:32.412929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03226e87-281d-444e-988b-5864440d4b7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:32.414755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03226e87-281d-444e-988b-5864440d4b7f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:32.436108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75deb7b-9901-4d64-a650-d25e27030e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:32.436924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75deb7b-9901-4d64-a650-d25e27030e39 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:32.437209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75deb7b-9901-4d64-a650-d25e27030e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:32.437487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75deb7b-9901-4d64-a650-d25e27030e39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:32.442753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75deb7b-9901-4d64-a650-d25e27030e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:32.444216 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75deb7b-9901-4d64-a650-d25e27030e39 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:32.445906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 74/145] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:32.451425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75deb7b-9901-4d64-a650-d25e27030e39 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:32.457428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:32.585163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 72/146] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 26 02:51:32.591314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03226e87-281d-444e-988b-5864440d4b7f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:32.593096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75deb7b-9901-4d64-a650-d25e27030e39 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:32.634253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f1bdc-2837-4055-b276-093d3314f034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:32.634938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1bdc-2837-4055-b276-093d3314f034 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:32.635222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1bdc-2837-4055-b276-093d3314f034 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:32.635501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f1bdc-2837-4055-b276-093d3314f034 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:32.636238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f1bdc-2837-4055-b276-093d3314f034 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:32.637750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389f1bdc-2837-4055-b276-093d3314f034 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:32.650405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:32.681797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 75/147] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:51:32 2022] POST /identity/v3/users => generated 283 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:51:32.759334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03226e87-281d-444e-988b-5864440d4b7f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:32.807539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872a02d-2036-4d07-b34e-35b59ad733a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:32.808424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872a02d-2036-4d07-b34e-35b59ad733a2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:32.809275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872a02d-2036-4d07-b34e-35b59ad733a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:32.809557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872a02d-2036-4d07-b34e-35b59ad733a2 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:32.814780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872a02d-2036-4d07-b34e-35b59ad733a2 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:32.816252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872a02d-2036-4d07-b34e-35b59ad733a2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:32.818373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 73/148] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:32 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:32.824273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f1bdc-2837-4055-b276-093d3314f034 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:32.933039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:32.985887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389f1bdc-2837-4055-b276-093d3314f034 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:33.037283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad4dfda-7cfa-49bc-a180-ac01b5855879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:33.037969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4dfda-7cfa-49bc-a180-ac01b5855879 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:33.038301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4dfda-7cfa-49bc-a180-ac01b5855879 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:33.038582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad4dfda-7cfa-49bc-a180-ac01b5855879 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:33.039415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4dfda-7cfa-49bc-a180-ac01b5855879 None admin] RBAC: Adding query filter params (role.id=e1ed407e2d924fac8012250bb2ecf94f, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c, user.id=19c972280f644c03b032e16fcb039658) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:33.039732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4dfda-7cfa-49bc-a180-ac01b5855879 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:33.041194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad4dfda-7cfa-49bc-a180-ac01b5855879 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:33.047895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 76/149] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:32 2022] GET /identity/v3/role_assignments?user.id=19c972280f644c03b032e16fcb039658&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:33.059140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b19037-1ede-4f60-84d9-17e429f226c7 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:33.059692 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b19037-1ede-4f60-84d9-17e429f226c7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:33.060125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b19037-1ede-4f60-84d9-17e429f226c7 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:33.061695 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 74/150] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:33.067374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1f22c-fc55-42dd-b9d7-70fdeec2ef68 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:33.067896 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1f22c-fc55-42dd-b9d7-70fdeec2ef68 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:33.068339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1f22c-fc55-42dd-b9d7-70fdeec2ef68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:33.095321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-eac1f22c-fc55-42dd-b9d7-70fdeec2ef68 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:33.096588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eac1f22c-fc55-42dd-b9d7-70fdeec2ef68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:33.188575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 77/151] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:33.222843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b19037-1ede-4f60-84d9-17e429f226c7 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:33.347165 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:33.358484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b19037-1ede-4f60-84d9-17e429f226c7 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:33.405706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015c09e-90f3-4b8b-baa5-03872abb62a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:33.406550 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015c09e-90f3-4b8b-baa5-03872abb62a8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:33.406841 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015c09e-90f3-4b8b-baa5-03872abb62a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:33.407120 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015c09e-90f3-4b8b-baa5-03872abb62a8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:33.420562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e015c09e-90f3-4b8b-baa5-03872abb62a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e015c09e-90f3-4b8b-baa5-03872abb62a8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:33.428597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:33.429470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:33.430274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 75/152] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:33.437389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1f22c-fc55-42dd-b9d7-70fdeec2ef68 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:33.500843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:33.550832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac1f22c-fc55-42dd-b9d7-70fdeec2ef68 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:33.587565 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37406c78-4199-4037-abe8-2423dd5c9061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:33.588264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37406c78-4199-4037-abe8-2423dd5c9061 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:33.588549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37406c78-4199-4037-abe8-2423dd5c9061 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:33.588825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37406c78-4199-4037-abe8-2423dd5c9061 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:33.589648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37406c78-4199-4037-abe8-2423dd5c9061 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:33.590558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37406c78-4199-4037-abe8-2423dd5c9061 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:33.591972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37406c78-4199-4037-abe8-2423dd5c9061 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:33.601553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 78/153] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:33.606623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015c09e-90f3-4b8b-baa5-03872abb62a8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:33.689922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddf8ea-764c-4baf-b459-9152e430d102 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:33.690328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddf8ea-764c-4baf-b459-9152e430d102 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:33.690711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ddf8ea-764c-4baf-b459-9152e430d102 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:33.692423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 79/154] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:33.697425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8690-fab1-4521-9987-cda754e3f079 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:33.697720 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8690-fab1-4521-9987-cda754e3f079 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:33.697999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a8690-fab1-4521-9987-cda754e3f079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:33.731149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015c09e-90f3-4b8b-baa5-03872abb62a8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:33.731642 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2a5a8690-fab1-4521-9987-cda754e3f079 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:33.732857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2a5a8690-fab1-4521-9987-cda754e3f079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:33.791180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d12d7f-a7d5-44de-bd93-20ec6b99d759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:33.792000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d12d7f-a7d5-44de-bd93-20ec6b99d759 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:33.792287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d12d7f-a7d5-44de-bd93-20ec6b99d759 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:33.792563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d12d7f-a7d5-44de-bd93-20ec6b99d759 None admin] PATH_INFO: `/v3/users/581bd67ace7c43e5a7d3bb35a0182692` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:33.798872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d12d7f-a7d5-44de-bd93-20ec6b99d759 None admin] RBAC: Authorizing `identity:get_user(user_id=581bd67ace7c43e5a7d3bb35a0182692)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:33.800344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d12d7f-a7d5-44de-bd93-20ec6b99d759 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:33.802233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 76/155] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:33 2022] GET /identity/v3/users/581bd67ace7c43e5a7d3bb35a0182692 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:33.807730 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d12d7f-a7d5-44de-bd93-20ec6b99d759 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:33.835114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 80/156] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:33.856889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:33.872199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a8690-fab1-4521-9987-cda754e3f079 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:33.979273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:33.990780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d12d7f-a7d5-44de-bd93-20ec6b99d759 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:34.031510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dd16a-a0af-4d89-803e-4f4458352790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:34.032704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dd16a-a0af-4d89-803e-4f4458352790 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.033210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dd16a-a0af-4d89-803e-4f4458352790 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.033691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dd16a-a0af-4d89-803e-4f4458352790 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.035149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55dd16a-a0af-4d89-803e-4f4458352790 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:34.036804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55dd16a-a0af-4d89-803e-4f4458352790 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:34.038769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 77/157] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:33 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:34.045048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dd16a-a0af-4d89-803e-4f4458352790 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:34.097580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a8690-fab1-4521-9987-cda754e3f079 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:34.159469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a703a79-946d-460d-ae22-f19201f4a28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:34.160275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a703a79-946d-460d-ae22-f19201f4a28d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.160553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a703a79-946d-460d-ae22-f19201f4a28d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.160825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a703a79-946d-460d-ae22-f19201f4a28d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.170071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:34.179872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dd16a-a0af-4d89-803e-4f4458352790 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:34.189424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a703a79-946d-460d-ae22-f19201f4a28d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0a703a79-946d-460d-ae22-f19201f4a28d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:34.197566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:34.198551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:34.199931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 81/158] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:51:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:34.206867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a703a79-946d-460d-ae22-f19201f4a28d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:34.221053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f2639-a158-4e86-8dc8-5da5c81c7dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:34.222323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f2639-a158-4e86-8dc8-5da5c81c7dfd None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.222821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f2639-a158-4e86-8dc8-5da5c81c7dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.223289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f2639-a158-4e86-8dc8-5da5c81c7dfd None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/users/581bd67ace7c43e5a7d3bb35a0182692/roles/c2c12c04e72a4116bf5d8e58c2cedc7b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.225678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f2639-a158-4e86-8dc8-5da5c81c7dfd None admin] RBAC: Authorizing `identity:create_grant(project_id=e855e8bdc61c4a8a91cadb04fe89906c, user_id=581bd67ace7c43e5a7d3bb35a0182692, role_id=c2c12c04e72a4116bf5d8e58c2cedc7b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:34.227389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f2639-a158-4e86-8dc8-5da5c81c7dfd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:34.241938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 78/159] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:51:34 2022] PUT /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/users/581bd67ace7c43e5a7d3bb35a0182692/roles/c2c12c04e72a4116bf5d8e58c2cedc7b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:51:34.313732 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369267c-9880-439c-bd65-6621ae4576a8 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.314365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369267c-9880-439c-bd65-6621ae4576a8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.314868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2369267c-9880-439c-bd65-6621ae4576a8 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.316817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 79/160] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:34.324199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290b901-8212-471d-addd-64357689196c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.324818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290b901-8212-471d-addd-64357689196c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.325449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7290b901-8212-471d-addd-64357689196c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.345768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a703a79-946d-460d-ae22-f19201f4a28d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:34.348019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7290b901-8212-471d-addd-64357689196c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:34.349583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7290b901-8212-471d-addd-64357689196c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:34.354336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:34.414788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbb4e9-d914-40e2-bc78-f34298dbcc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:34.415485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbb4e9-d914-40e2-bc78-f34298dbcc3e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.415795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbb4e9-d914-40e2-bc78-f34298dbcc3e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.416052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbbb4e9-d914-40e2-bc78-f34298dbcc3e None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.416820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbb4e9-d914-40e2-bc78-f34298dbcc3e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:34.417134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbb4e9-d914-40e2-bc78-f34298dbcc3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:34.419275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbbb4e9-d914-40e2-bc78-f34298dbcc3e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:34.430901 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 80/161] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:34.441544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 82/162] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:34.447087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290b901-8212-471d-addd-64357689196c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:34.480297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbb4e9-d914-40e2-bc78-f34298dbcc3e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:34.542434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:34.563683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7290b901-8212-471d-addd-64357689196c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:34.604683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542da97-58b5-4bb8-b4c0-95d8efcfe896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:34.606643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542da97-58b5-4bb8-b4c0-95d8efcfe896 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.607761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542da97-58b5-4bb8-b4c0-95d8efcfe896 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.608260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8542da97-58b5-4bb8-b4c0-95d8efcfe896 None admin] PATH_INFO: `/v3/users/a05c37bdf694480a8a14de0b18b2da4b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.609764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542da97-58b5-4bb8-b4c0-95d8efcfe896 None admin] RBAC: Authorizing `identity:get_user(user_id=a05c37bdf694480a8a14de0b18b2da4b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:34.611500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8542da97-58b5-4bb8-b4c0-95d8efcfe896 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:34.613468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 81/163] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:34 2022] GET /identity/v3/users/a05c37bdf694480a8a14de0b18b2da4b => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:34.619273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542da97-58b5-4bb8-b4c0-95d8efcfe896 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:34.695268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbbb4e9-d914-40e2-bc78-f34298dbcc3e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:34.745479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7b8768-2b85-40fc-8c49-8d9633062479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:34.747420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b8768-2b85-40fc-8c49-8d9633062479 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.748106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b8768-2b85-40fc-8c49-8d9633062479 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.748839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7b8768-2b85-40fc-8c49-8d9633062479 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.759541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8542da97-58b5-4bb8-b4c0-95d8efcfe896 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:34.766956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7b8768-2b85-40fc-8c49-8d9633062479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3a7b8768-2b85-40fc-8c49-8d9633062479 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:34.778127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:34.779024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:34.779771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:34.782395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 83/164] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:51:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:34.784230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c26f-05c5-409c-80b4-05d39d654f46 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.784911 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c26f-05c5-409c-80b4-05d39d654f46 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.785452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb63c26f-05c5-409c-80b4-05d39d654f46 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.787008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 84/165] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:34.788422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c26f-05c5-409c-80b4-05d39d654f46 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:34.802952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9186b-9961-4965-a1af-fb5eaab6b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:34.803615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9186b-9961-4965-a1af-fb5eaab6b0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.803900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9186b-9961-4965-a1af-fb5eaab6b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.804493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac9186b-9961-4965-a1af-fb5eaab6b0b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.826322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:34.866463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:34.922904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb63c26f-05c5-409c-80b4-05d39d654f46 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:34.957787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9186b-9961-4965-a1af-fb5eaab6b0b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:34.959304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac9186b-9961-4965-a1af-fb5eaab6b0b0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:34.961468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75901ad0-9ee1-47bc-b059-bdd8898ee87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:34.962165 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75901ad0-9ee1-47bc-b059-bdd8898ee87e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.962501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75901ad0-9ee1-47bc-b059-bdd8898ee87e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.962779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75901ad0-9ee1-47bc-b059-bdd8898ee87e None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.963534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75901ad0-9ee1-47bc-b059-bdd8898ee87e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:34.963840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75901ad0-9ee1-47bc-b059-bdd8898ee87e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:34.965772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75901ad0-9ee1-47bc-b059-bdd8898ee87e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:34.974661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 85/166] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:51:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:34.976138 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051e66b-4c35-40c8-9551-dec435ffa322 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:34.976429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051e66b-4c35-40c8-9551-dec435ffa322 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:34.976704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5051e66b-4c35-40c8-9551-dec435ffa322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:34.994344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5051e66b-4c35-40c8-9551-dec435ffa322 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:34.995527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5051e66b-4c35-40c8-9551-dec435ffa322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:35.063248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 82/167] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 26 02:51:35.064023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9186b-9961-4965-a1af-fb5eaab6b0b0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:35.094980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 86/168] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:35.095837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051e66b-4c35-40c8-9551-dec435ffa322 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:35.126618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:35.154385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:35.166729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac9186b-9961-4965-a1af-fb5eaab6b0b0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:35.201950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcde21-103c-44cd-9210-40a69485a00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:35.202894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcde21-103c-44cd-9210-40a69485a00b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:35.203184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcde21-103c-44cd-9210-40a69485a00b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:35.203468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcde21-103c-44cd-9210-40a69485a00b None admin] PATH_INFO: `/v3/users/19c972280f644c03b032e16fcb039658` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:35.210857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcde21-103c-44cd-9210-40a69485a00b None admin] RBAC: Authorizing `identity:get_user(user_id=19c972280f644c03b032e16fcb039658)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:35.212589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dcde21-103c-44cd-9210-40a69485a00b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:35.217161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 83/169] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:35 2022] GET /identity/v3/users/19c972280f644c03b032e16fcb039658 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:35.222263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcde21-103c-44cd-9210-40a69485a00b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:35.271175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5051e66b-4c35-40c8-9551-dec435ffa322 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:35.344045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2963b-0aa3-4cff-8236-d0571dbd56e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:35.344926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2963b-0aa3-4cff-8236-d0571dbd56e3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:35.345213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2963b-0aa3-4cff-8236-d0571dbd56e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:35.345484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2963b-0aa3-4cff-8236-d0571dbd56e3 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:35.346862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2963b-0aa3-4cff-8236-d0571dbd56e3 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:35.348306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2963b-0aa3-4cff-8236-d0571dbd56e3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:35.350245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 87/170] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:35 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:35.355573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2963b-0aa3-4cff-8236-d0571dbd56e3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:35.430394 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dcde21-103c-44cd-9210-40a69485a00b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:35.471116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:35.512167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d1993-41ec-47ac-959b-08e44c9c6124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:35.512983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d1993-41ec-47ac-959b-08e44c9c6124 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:35.513268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d1993-41ec-47ac-959b-08e44c9c6124 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:35.513546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0d1993-41ec-47ac-959b-08e44c9c6124 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:35.542964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0d1993-41ec-47ac-959b-08e44c9c6124 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-cd0d1993-41ec-47ac-959b-08e44c9c6124 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:35.550746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:35.551590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:35.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 84/171] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:35.559100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:35.559556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d1993-41ec-47ac-959b-08e44c9c6124 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:35.596158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2963b-0aa3-4cff-8236-d0571dbd56e3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:35.690827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06b9c0-ce56-41c2-bcc4-dcc7229ed26f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:35.691403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06b9c0-ce56-41c2-bcc4-dcc7229ed26f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:35.691681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06b9c0-ce56-41c2-bcc4-dcc7229ed26f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:35.691952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e06b9c0-ce56-41c2-bcc4-dcc7229ed26f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:35.826304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0d1993-41ec-47ac-959b-08e44c9c6124 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:35.908118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06b9c0-ce56-41c2-bcc4-dcc7229ed26f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:35.909595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e06b9c0-ce56-41c2-bcc4-dcc7229ed26f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:35.966844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cca52-9f43-468e-8af9-2f0970d19a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:35.967449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cca52-9f43-468e-8af9-2f0970d19a06 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:35.967745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cca52-9f43-468e-8af9-2f0970d19a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:35.968029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0cca52-9f43-468e-8af9-2f0970d19a06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:35.968889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cca52-9f43-468e-8af9-2f0970d19a06 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c, user.id=a05c37bdf694480a8a14de0b18b2da4b) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:35.969211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cca52-9f43-468e-8af9-2f0970d19a06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:35.970719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0cca52-9f43-468e-8af9-2f0970d19a06 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:35.988442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 85/172] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:35 2022] GET /identity/v3/role_assignments?user.id=a05c37bdf694480a8a14de0b18b2da4b&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 608 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:35.989100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cca52-9f43-468e-8af9-2f0970d19a06 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:36.102416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:36.138853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:36.181190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 88/173] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 830 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:36.187516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06b9c0-ce56-41c2-bcc4-dcc7229ed26f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:36.280453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0cca52-9f43-468e-8af9-2f0970d19a06 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:36.375359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094871a5-a200-4edd-af9a-9f93315b3a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:36.376049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094871a5-a200-4edd-af9a-9f93315b3a69 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:36.376337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094871a5-a200-4edd-af9a-9f93315b3a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:36.376614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094871a5-a200-4edd-af9a-9f93315b3a69 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:36.377379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094871a5-a200-4edd-af9a-9f93315b3a69 None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:36.377691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094871a5-a200-4edd-af9a-9f93315b3a69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:36.379098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094871a5-a200-4edd-af9a-9f93315b3a69 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:36.398036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 86/174] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:36.405589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094871a5-a200-4edd-af9a-9f93315b3a69 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:36.468290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e06b9c0-ce56-41c2-bcc4-dcc7229ed26f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:36.559402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901b355-f553-4077-975c-e4b1ff44c4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:36.560214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901b355-f553-4077-975c-e4b1ff44c4bf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:36.560495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901b355-f553-4077-975c-e4b1ff44c4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:36.560766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b901b355-f553-4077-975c-e4b1ff44c4bf None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:36.562032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901b355-f553-4077-975c-e4b1ff44c4bf None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:36.563518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b901b355-f553-4077-975c-e4b1ff44c4bf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:36.565420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 89/175] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:36 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:36.571697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901b355-f553-4077-975c-e4b1ff44c4bf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:36.668792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094871a5-a200-4edd-af9a-9f93315b3a69 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:36.686106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:36.711387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:36.745450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733718f8-7ef1-40f6-8f87-adf2321952e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:36.746639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733718f8-7ef1-40f6-8f87-adf2321952e1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:36.747100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733718f8-7ef1-40f6-8f87-adf2321952e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:36.747534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733718f8-7ef1-40f6-8f87-adf2321952e1 None admin] PATH_INFO: `/v3/users/19c972280f644c03b032e16fcb039658` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:36.749151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733718f8-7ef1-40f6-8f87-adf2321952e1 None admin] RBAC: Authorizing `identity:get_user(user_id=19c972280f644c03b032e16fcb039658)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:36.752803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733718f8-7ef1-40f6-8f87-adf2321952e1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:36.754873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 87/176] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:36 2022] GET /identity/v3/users/19c972280f644c03b032e16fcb039658 => generated 275 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:36.756286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2c369-91f1-4f9e-a919-4dca8051978f None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:36.756698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2c369-91f1-4f9e-a919-4dca8051978f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:36.757199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2c369-91f1-4f9e-a919-4dca8051978f None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:36.758597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 88/177] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:36.764353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601daa-ab96-4241-8aeb-35a26c00b7c1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:36.765072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601daa-ab96-4241-8aeb-35a26c00b7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:36.765072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9601daa-ab96-4241-8aeb-35a26c00b7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:36.779531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b901b355-f553-4077-975c-e4b1ff44c4bf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:36.795024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f9601daa-ab96-4241-8aeb-35a26c00b7c1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:36.796426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f9601daa-ab96-4241-8aeb-35a26c00b7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:36.823652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354cc4f-d21c-47da-8289-a20434558806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:36.824749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354cc4f-d21c-47da-8289-a20434558806 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:36.825029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354cc4f-d21c-47da-8289-a20434558806 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:36.825298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354cc4f-d21c-47da-8289-a20434558806 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:36.826157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354cc4f-d21c-47da-8289-a20434558806 None admin] RBAC: Adding query filter params (role.id=4f31bdaad0194bbd8fab39617059d14a, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c, user.id=19c972280f644c03b032e16fcb039658) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:36.829806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354cc4f-d21c-47da-8289-a20434558806 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:36.832588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354cc4f-d21c-47da-8289-a20434558806 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:36.845873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 90/178] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:36 2022] GET /identity/v3/role_assignments?user.id=19c972280f644c03b032e16fcb039658&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=4f31bdaad0194bbd8fab39617059d14a => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:36.857885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354cc4f-d21c-47da-8289-a20434558806 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:36.939661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 89/179] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:36.980524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9601daa-ab96-4241-8aeb-35a26c00b7c1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:36.988899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354cc4f-d21c-47da-8289-a20434558806 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:37.012755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:37.039858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed08899-7582-4959-8823-d26202b5f1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:37.040921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed08899-7582-4959-8823-d26202b5f1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.041412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed08899-7582-4959-8823-d26202b5f1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.041890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed08899-7582-4959-8823-d26202b5f1a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.114161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9601daa-ab96-4241-8aeb-35a26c00b7c1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:37.114161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:37.154014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed08899-7582-4959-8823-d26202b5f1a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:37.155898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed08899-7582-4959-8823-d26202b5f1a6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:37.160663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7abf46-386c-4fee-bed6-a6887d8e3c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:37.161474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7abf46-386c-4fee-bed6-a6887d8e3c7a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.161755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7abf46-386c-4fee-bed6-a6887d8e3c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.162027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7abf46-386c-4fee-bed6-a6887d8e3c7a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.177098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7abf46-386c-4fee-bed6-a6887d8e3c7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-3a7abf46-386c-4fee-bed6-a6887d8e3c7a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:37.185296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:37.186773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:37.188200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 90/180] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:51:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:37.188870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe8572-20f1-4b7e-bf3c-4a484cbecae8 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.188870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe8572-20f1-4b7e-bf3c-4a484cbecae8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.188870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe8572-20f1-4b7e-bf3c-4a484cbecae8 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.190461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 91/181] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:37.197400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423389b1-03fa-4fa6-928a-558f5536e65a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.197691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423389b1-03fa-4fa6-928a-558f5536e65a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.197971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423389b1-03fa-4fa6-928a-558f5536e65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.218399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-423389b1-03fa-4fa6-928a-558f5536e65a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:37.219995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-423389b1-03fa-4fa6-928a-558f5536e65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:37.273967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 91/182] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:37.275870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed08899-7582-4959-8823-d26202b5f1a6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:37.299657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:37.299957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 92/183] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:37.301257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f5303-cf45-4879-af51-2b453447cd27 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.301700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f5303-cf45-4879-af51-2b453447cd27 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.302170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077f5303-cf45-4879-af51-2b453447cd27 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.303425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 93/184] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:37.304214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f5303-cf45-4879-af51-2b453447cd27 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:37.396103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed08899-7582-4959-8823-d26202b5f1a6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:37.410162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:37.420515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077f5303-cf45-4879-af51-2b453447cd27 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:37.433003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c03f297-3e73-4e4b-a622-b3e0a98e979c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:37.433874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03f297-3e73-4e4b-a622-b3e0a98e979c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.434198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03f297-3e73-4e4b-a622-b3e0a98e979c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.434976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c03f297-3e73-4e4b-a622-b3e0a98e979c None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.436132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c03f297-3e73-4e4b-a622-b3e0a98e979c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:37.436247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c03f297-3e73-4e4b-a622-b3e0a98e979c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:37.438087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c03f297-3e73-4e4b-a622-b3e0a98e979c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:37.447386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 92/185] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:37.453643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d779d-24ad-480f-a2e9-c1a6c016a398 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.454113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d779d-24ad-480f-a2e9-c1a6c016a398 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.454555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d779d-24ad-480f-a2e9-c1a6c016a398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.456431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ec8e-4c22-49ef-bc3f-fb0983463b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:37.457294 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ec8e-4c22-49ef-bc3f-fb0983463b3b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.457581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ec8e-4c22-49ef-bc3f-fb0983463b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.457856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb59ec8e-4c22-49ef-bc3f-fb0983463b3b None admin] PATH_INFO: `/v3/projects/fd662c840e364e918c1973e41dd5aa73` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.459080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59ec8e-4c22-49ef-bc3f-fb0983463b3b None admin] RBAC: Authorizing `identity:get_project(project_id=fd662c840e364e918c1973e41dd5aa73)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:37.460703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb59ec8e-4c22-49ef-bc3f-fb0983463b3b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:37.462470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 94/186] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:37 2022] GET /identity/v3/projects/fd662c840e364e918c1973e41dd5aa73 => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:37.462470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ec8e-4c22-49ef-bc3f-fb0983463b3b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:37.476040 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0e7d779d-24ad-480f-a2e9-c1a6c016a398 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:37.477240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0e7d779d-24ad-480f-a2e9-c1a6c016a398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:37.546869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 93/187] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:37.555319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d779d-24ad-480f-a2e9-c1a6c016a398 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:37.570101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:37.571919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb59ec8e-4c22-49ef-bc3f-fb0983463b3b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:37.606689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d48b2-6aaa-450a-9d39-1fe614ec9dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:37.607488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d48b2-6aaa-450a-9d39-1fe614ec9dbb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.607772 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d48b2-6aaa-450a-9d39-1fe614ec9dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.608050 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d48b2-6aaa-450a-9d39-1fe614ec9dbb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.609222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d48b2-6aaa-450a-9d39-1fe614ec9dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:37.610699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d48b2-6aaa-450a-9d39-1fe614ec9dbb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:37.612350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 95/188] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:37.613196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d48b2-6aaa-450a-9d39-1fe614ec9dbb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:37.640858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:37.657762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d779d-24ad-480f-a2e9-c1a6c016a398 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:37.690690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd622a-c701-4ba5-96e7-004655ef8906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:37.691557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd622a-c701-4ba5-96e7-004655ef8906 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.691834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd622a-c701-4ba5-96e7-004655ef8906 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.692104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd622a-c701-4ba5-96e7-004655ef8906 None admin] PATH_INFO: `/v3/users/3e3cce5b2efb4a6eaaefb687f866d021` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.705512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd622a-c701-4ba5-96e7-004655ef8906 None admin] RBAC: Authorizing `identity:get_user(user_id=3e3cce5b2efb4a6eaaefb687f866d021)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:37.707000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd622a-c701-4ba5-96e7-004655ef8906 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:37.708668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 94/189] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:37 2022] GET /identity/v3/users/3e3cce5b2efb4a6eaaefb687f866d021 => generated 283 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:37.710973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766c0b-7e2e-4635-8ea0-bce42927b04e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.711262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766c0b-7e2e-4635-8ea0-bce42927b04e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.711542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc766c0b-7e2e-4635-8ea0-bce42927b04e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.712601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d48b2-6aaa-450a-9d39-1fe614ec9dbb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:37.714073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 95/190] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:37.714073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc766c0b-7e2e-4635-8ea0-bce42927b04e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:37.745768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0dec0-8ae2-4ece-90bf-69cbf09a5323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:37.746957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0dec0-8ae2-4ece-90bf-69cbf09a5323 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.747238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0dec0-8ae2-4ece-90bf-69cbf09a5323 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.747510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0dec0-8ae2-4ece-90bf-69cbf09a5323 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.748593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0dec0-8ae2-4ece-90bf-69cbf09a5323 None admin] RBAC: Adding query filter params (role.id=e1ed407e2d924fac8012250bb2ecf94f, scope.project.id=fd662c840e364e918c1973e41dd5aa73, user.id=19c972280f644c03b032e16fcb039658) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:37.748901 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0dec0-8ae2-4ece-90bf-69cbf09a5323 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:37.750339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0dec0-8ae2-4ece-90bf-69cbf09a5323 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:37.756036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 96/191] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:37 2022] GET /identity/v3/role_assignments?user.id=19c972280f644c03b032e16fcb039658&scope.project.id=fd662c840e364e918c1973e41dd5aa73&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:37.756780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0dec0-8ae2-4ece-90bf-69cbf09a5323 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:37.798464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:37.834832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc766c0b-7e2e-4635-8ea0-bce42927b04e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:37.862545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:37.874863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0dec0-8ae2-4ece-90bf-69cbf09a5323 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:37.877592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0b257-c3b0-4554-a8ae-f53f879eae2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:37.878461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0b257-c3b0-4554-a8ae-f53f879eae2c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.878739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0b257-c3b0-4554-a8ae-f53f879eae2c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.879012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea0b257-c3b0-4554-a8ae-f53f879eae2c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.891148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea0b257-c3b0-4554-a8ae-f53f879eae2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9ea0b257-c3b0-4554-a8ae-f53f879eae2c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:37.900054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:37.901190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 96/192] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:51:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:37.902739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0b257-c3b0-4554-a8ae-f53f879eae2c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:37.919312 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0c53d-6272-4567-a5fe-164b4e8c643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:37.919940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0c53d-6272-4567-a5fe-164b4e8c643c None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.920213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0c53d-6272-4567-a5fe-164b4e8c643c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.920491 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0c53d-6272-4567-a5fe-164b4e8c643c None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.921923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0c53d-6272-4567-a5fe-164b4e8c643c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:37.923134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0c53d-6272-4567-a5fe-164b4e8c643c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:37.943162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 97/193] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:51:37 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:51:37.944391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd31001-07a4-4a84-ba96-b0260fd39479 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:37.944715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd31001-07a4-4a84-ba96-b0260fd39479 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:37.944955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd31001-07a4-4a84-ba96-b0260fd39479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:37.963201 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bbd31001-07a4-4a84-ba96-b0260fd39479 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:37.965227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bbd31001-07a4-4a84-ba96-b0260fd39479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:38.039952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea0b257-c3b0-4554-a8ae-f53f879eae2c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:38.066621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 98/194] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:38.066621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd31001-07a4-4a84-ba96-b0260fd39479 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:38.077778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:38.086563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7284c-17cc-43b1-87bd-5fb72f9e4c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:38.087225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7284c-17cc-43b1-87bd-5fb72f9e4c59 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:38.087508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7284c-17cc-43b1-87bd-5fb72f9e4c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:38.087780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7284c-17cc-43b1-87bd-5fb72f9e4c59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:38.165857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7284c-17cc-43b1-87bd-5fb72f9e4c59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:38.166761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:38.171195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7284c-17cc-43b1-87bd-5fb72f9e4c59 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:38.199431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd31001-07a4-4a84-ba96-b0260fd39479 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:38.246791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e254f3-e0b3-42f4-8d4d-43d35ab2e1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:38.247480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e254f3-e0b3-42f4-8d4d-43d35ab2e1ce None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:38.247766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e254f3-e0b3-42f4-8d4d-43d35ab2e1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:38.248043 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e254f3-e0b3-42f4-8d4d-43d35ab2e1ce None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:38.249134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e254f3-e0b3-42f4-8d4d-43d35ab2e1ce None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:38.249446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e254f3-e0b3-42f4-8d4d-43d35ab2e1ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:38.250881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e254f3-e0b3-42f4-8d4d-43d35ab2e1ce None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:38.260902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 99/195] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:51:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:38.261610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e254f3-e0b3-42f4-8d4d-43d35ab2e1ce None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:38.296511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 97/196] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:38.297376 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7284c-17cc-43b1-87bd-5fb72f9e4c59 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:38.336181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:38.385736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e254f3-e0b3-42f4-8d4d-43d35ab2e1ce None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:38.408260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7284c-17cc-43b1-87bd-5fb72f9e4c59 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:38.424058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391155b-a4f9-42b9-9ca5-86b97dd8937d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:38.425190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391155b-a4f9-42b9-9ca5-86b97dd8937d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:38.425678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391155b-a4f9-42b9-9ca5-86b97dd8937d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:38.426299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5391155b-a4f9-42b9-9ca5-86b97dd8937d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:38.439454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5391155b-a4f9-42b9-9ca5-86b97dd8937d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:38.444842 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f82d6a-0b13-4e4b-940a-6e89c52656b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:38.445711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f82d6a-0b13-4e4b-940a-6e89c52656b6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:38.445996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f82d6a-0b13-4e4b-940a-6e89c52656b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:38.446312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f82d6a-0b13-4e4b-940a-6e89c52656b6 None admin] PATH_INFO: `/v3/users/3e3cce5b2efb4a6eaaefb687f866d021` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:38.447511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f82d6a-0b13-4e4b-940a-6e89c52656b6 None admin] RBAC: Authorizing `identity:get_user(user_id=3e3cce5b2efb4a6eaaefb687f866d021)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:38.448955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f82d6a-0b13-4e4b-940a-6e89c52656b6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:38.450424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:38.451473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 98/197] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:38 2022] GET /identity/v3/users/3e3cce5b2efb4a6eaaefb687f866d021 => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:38.454166 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f82d6a-0b13-4e4b-940a-6e89c52656b6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5391155b-a4f9-42b9-9ca5-86b97dd8937d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:38.458980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:38.461629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:38.462586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:38.464785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 100/198] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:38.464785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391155b-a4f9-42b9-9ca5-86b97dd8937d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:38.556408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:38.560888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f82d6a-0b13-4e4b-940a-6e89c52656b6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:38.577157 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5391155b-a4f9-42b9-9ca5-86b97dd8937d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:38.597202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4661f5-0a1e-43d6-a590-fe014c4e433b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:38.598201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4661f5-0a1e-43d6-a590-fe014c4e433b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:38.598554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4661f5-0a1e-43d6-a590-fe014c4e433b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:38.598897 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4661f5-0a1e-43d6-a590-fe014c4e433b None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:38.600167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4661f5-0a1e-43d6-a590-fe014c4e433b None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:38.601678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4661f5-0a1e-43d6-a590-fe014c4e433b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:38.603520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 99/199] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:38 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:38.608997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4661f5-0a1e-43d6-a590-fe014c4e433b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:38.614276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68b62b-d3eb-4458-8885-d2d8e882bc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:38.614934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68b62b-d3eb-4458-8885-d2d8e882bc38 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:38.615222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68b62b-d3eb-4458-8885-d2d8e882bc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:38.615501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc68b62b-d3eb-4458-8885-d2d8e882bc38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:38.701677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68b62b-d3eb-4458-8885-d2d8e882bc38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:38.703818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc68b62b-d3eb-4458-8885-d2d8e882bc38 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:38.708817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:38.721568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4661f5-0a1e-43d6-a590-fe014c4e433b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:38.758496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40279c-a884-40c7-83e2-41becea70b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:38.759471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40279c-a884-40c7-83e2-41becea70b49 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:38.759924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40279c-a884-40c7-83e2-41becea70b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:38.760564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40279c-a884-40c7-83e2-41becea70b49 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:38.762909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40279c-a884-40c7-83e2-41becea70b49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:38.763778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40279c-a884-40c7-83e2-41becea70b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:38.766652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40279c-a884-40c7-83e2-41becea70b49 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:38.778373 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 100/200] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:38.783437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40279c-a884-40c7-83e2-41becea70b49 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:38.815683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:38.832260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 101/201] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:38.833687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68b62b-d3eb-4458-8885-d2d8e882bc38 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:38.900737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40279c-a884-40c7-83e2-41becea70b49 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:38.970949 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12326a3d-1b44-4baa-b127-66f8489fc1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:38.971549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12326a3d-1b44-4baa-b127-66f8489fc1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:38.971832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12326a3d-1b44-4baa-b127-66f8489fc1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:38.972105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12326a3d-1b44-4baa-b127-66f8489fc1f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:38.972951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12326a3d-1b44-4baa-b127-66f8489fc1f0 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=65124292b1924cc889516053c00bda74, user.id=3e3cce5b2efb4a6eaaefb687f866d021) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:38.973267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12326a3d-1b44-4baa-b127-66f8489fc1f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:38.974735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12326a3d-1b44-4baa-b127-66f8489fc1f0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:38.982299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 101/202] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:38 2022] GET /identity/v3/role_assignments?user.id=3e3cce5b2efb4a6eaaefb687f866d021&scope.project.id=65124292b1924cc889516053c00bda74&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:38.983066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12326a3d-1b44-4baa-b127-66f8489fc1f0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:39.001234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:39.006733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc68b62b-d3eb-4458-8885-d2d8e882bc38 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:39.080300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2a17a-6102-4df5-b6a5-38def4a4fe8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:39.081183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2a17a-6102-4df5-b6a5-38def4a4fe8d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:39.081466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2a17a-6102-4df5-b6a5-38def4a4fe8d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:39.081742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2a17a-6102-4df5-b6a5-38def4a4fe8d None admin] PATH_INFO: `/v3/users/581bd67ace7c43e5a7d3bb35a0182692` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:39.084648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2a17a-6102-4df5-b6a5-38def4a4fe8d None admin] RBAC: Authorizing `identity:get_user(user_id=581bd67ace7c43e5a7d3bb35a0182692)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:39.088580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2a17a-6102-4df5-b6a5-38def4a4fe8d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:39.090912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 102/203] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:38 2022] GET /identity/v3/users/581bd67ace7c43e5a7d3bb35a0182692 => generated 289 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:39.098107 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2a17a-6102-4df5-b6a5-38def4a4fe8d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:39.237657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:39.262675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12326a3d-1b44-4baa-b127-66f8489fc1f0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:39.313730 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2a17a-6102-4df5-b6a5-38def4a4fe8d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:39.350030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf8064-8909-4be1-bd16-270c989d8edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:39.352716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bfab5-1d33-4953-825b-06b82ffec002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:39.354068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bfab5-1d33-4953-825b-06b82ffec002 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:39.354366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bfab5-1d33-4953-825b-06b82ffec002 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:39.354636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9bfab5-1d33-4953-825b-06b82ffec002 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:39.356595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf8064-8909-4be1-bd16-270c989d8edd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:39.363615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf8064-8909-4be1-bd16-270c989d8edd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:39.364217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbf8064-8909-4be1-bd16-270c989d8edd None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:39.365868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf8064-8909-4be1-bd16-270c989d8edd None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:39.371245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbf8064-8909-4be1-bd16-270c989d8edd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:39.380051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 102/204] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:38 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 397 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:39.386161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf8064-8909-4be1-bd16-270c989d8edd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:39.426412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:39.490769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9bfab5-1d33-4953-825b-06b82ffec002 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:39.492154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9bfab5-1d33-4953-825b-06b82ffec002 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:39.676478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbf8064-8909-4be1-bd16-270c989d8edd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:39.696852 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 103/205] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 26 02:51:39.702614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bfab5-1d33-4953-825b-06b82ffec002 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:39.755987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcbbf0-9d07-4e76-b1e5-c5e89ec918ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:39.756574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcbbf0-9d07-4e76-b1e5-c5e89ec918ea None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:39.756860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcbbf0-9d07-4e76-b1e5-c5e89ec918ea None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:39.757137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dcbbf0-9d07-4e76-b1e5-c5e89ec918ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:39.757979 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcbbf0-9d07-4e76-b1e5-c5e89ec918ea None admin] RBAC: Adding query filter params (role.id=4f31bdaad0194bbd8fab39617059d14a, scope.project.id=65124292b1924cc889516053c00bda74, user.id=3e3cce5b2efb4a6eaaefb687f866d021) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:39.758319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcbbf0-9d07-4e76-b1e5-c5e89ec918ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:39.759774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dcbbf0-9d07-4e76-b1e5-c5e89ec918ea None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:39.781763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 103/206] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:39 2022] GET /identity/v3/role_assignments?user.id=3e3cce5b2efb4a6eaaefb687f866d021&scope.project.id=65124292b1924cc889516053c00bda74&role.id=4f31bdaad0194bbd8fab39617059d14a => generated 259 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:39.808149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:39.846726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9bfab5-1d33-4953-825b-06b82ffec002 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:39.890823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b364e5-6234-454f-8345-b85b2d593dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:39.892082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b364e5-6234-454f-8345-b85b2d593dfb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:39.892628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b364e5-6234-454f-8345-b85b2d593dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:39.893101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b364e5-6234-454f-8345-b85b2d593dfb None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:39.894665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b364e5-6234-454f-8345-b85b2d593dfb None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:39.896395 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b364e5-6234-454f-8345-b85b2d593dfb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:39.898858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 104/207] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:39 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:39.905552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcbbf0-9d07-4e76-b1e5-c5e89ec918ea None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:39.934522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:40.030804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dcbbf0-9d07-4e76-b1e5-c5e89ec918ea None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:40.077281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a19076-09df-4a95-a1b8-68d99ec83697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:40.077958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19076-09df-4a95-a1b8-68d99ec83697 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:40.078263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19076-09df-4a95-a1b8-68d99ec83697 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:40.078539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a19076-09df-4a95-a1b8-68d99ec83697 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:40.079382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19076-09df-4a95-a1b8-68d99ec83697 None admin] RBAC: Adding query filter params (role.id=c2c12c04e72a4116bf5d8e58c2cedc7b, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c, user.id=581bd67ace7c43e5a7d3bb35a0182692) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:40.079695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19076-09df-4a95-a1b8-68d99ec83697 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:40.081143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a19076-09df-4a95-a1b8-68d99ec83697 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:40.091517 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 104/208] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:39 2022] GET /identity/v3/role_assignments?user.id=581bd67ace7c43e5a7d3bb35a0182692&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=c2c12c04e72a4116bf5d8e58c2cedc7b => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:40.265929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebad5-9576-4026-b0b6-88566ac58fff None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:40.266469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebad5-9576-4026-b0b6-88566ac58fff None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:40.266573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efebad5-9576-4026-b0b6-88566ac58fff None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:40.268040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 105/209] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:40.273602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b4bc6-6bef-41ac-90ee-8cb78c6d45b8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:40.273889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b4bc6-6bef-41ac-90ee-8cb78c6d45b8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:40.274190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b4bc6-6bef-41ac-90ee-8cb78c6d45b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:40.299154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-fd1b4bc6-6bef-41ac-90ee-8cb78c6d45b8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:40.300273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fd1b4bc6-6bef-41ac-90ee-8cb78c6d45b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:40.385129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 105/210] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:40.389985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474d09-fb44-422a-b40e-45f2869c80d9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:40.390540 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474d09-fb44-422a-b40e-45f2869c80d9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:40.391052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51474d09-fb44-422a-b40e-45f2869c80d9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:40.392952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 106/211] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:40.398017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b96d14-56cf-43f0-8c09-a2ffe57b932e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:40.398327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b96d14-56cf-43f0-8c09-a2ffe57b932e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:40.398604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b96d14-56cf-43f0-8c09-a2ffe57b932e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:40.416770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474d09-fb44-422a-b40e-45f2869c80d9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:40.417700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-12b96d14-56cf-43f0-8c09-a2ffe57b932e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:40.418971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-12b96d14-56cf-43f0-8c09-a2ffe57b932e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:40.438108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:40.503252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 106/212] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:40.530391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b96d14-56cf-43f0-8c09-a2ffe57b932e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:40.552832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51474d09-fb44-422a-b40e-45f2869c80d9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:40.593663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e95e34-9c92-478b-8bfc-c63f09da6b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:40.594651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e95e34-9c92-478b-8bfc-c63f09da6b7a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:40.594938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e95e34-9c92-478b-8bfc-c63f09da6b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:40.595288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e95e34-9c92-478b-8bfc-c63f09da6b7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:40.598229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:40.608932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e95e34-9c92-478b-8bfc-c63f09da6b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-79e95e34-9c92-478b-8bfc-c63f09da6b7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:40.619389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:40.620283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:40.621624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:40.622259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 107/213] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:40.629502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e95e34-9c92-478b-8bfc-c63f09da6b7a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:40.643365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b96d14-56cf-43f0-8c09-a2ffe57b932e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:40.686771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a7c82-5efe-4567-816e-316245fd718a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:40.687694 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a7c82-5efe-4567-816e-316245fd718a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:40.688030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a7c82-5efe-4567-816e-316245fd718a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:40.688522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8a7c82-5efe-4567-816e-316245fd718a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:40.689804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a7c82-5efe-4567-816e-316245fd718a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:40.691437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8a7c82-5efe-4567-816e-316245fd718a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:40.693288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 107/214] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:40.698703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a7c82-5efe-4567-816e-316245fd718a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:40.758435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:40.762231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e95e34-9c92-478b-8bfc-c63f09da6b7a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:40.805830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8a7c82-5efe-4567-816e-316245fd718a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:40.806316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efebd7c-8346-436a-a077-5d0f336ba0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:40.806991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efebd7c-8346-436a-a077-5d0f336ba0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:40.807274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efebd7c-8346-436a-a077-5d0f336ba0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:40.807548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efebd7c-8346-436a-a077-5d0f336ba0e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:40.808308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efebd7c-8346-436a-a077-5d0f336ba0e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:40.808647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efebd7c-8346-436a-a077-5d0f336ba0e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:40.811028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efebd7c-8346-436a-a077-5d0f336ba0e7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:40.821148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 108/215] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:40.826347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efebd7c-8346-436a-a077-5d0f336ba0e7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:40.852154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870b495-5d9c-4bf5-ace0-039285a70133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:40.853864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870b495-5d9c-4bf5-ace0-039285a70133 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:40.854483 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870b495-5d9c-4bf5-ace0-039285a70133 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:40.854961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870b495-5d9c-4bf5-ace0-039285a70133 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:40.856540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870b495-5d9c-4bf5-ace0-039285a70133 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:40.858263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870b495-5d9c-4bf5-ace0-039285a70133 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:40.889520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 108/216] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:51:40 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:51:40.910454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:40.954717 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efebd7c-8346-436a-a077-5d0f336ba0e7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:40.994193 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d8102-0ba7-4c11-a4e4-2e3a552b97eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:40.995394 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d8102-0ba7-4c11-a4e4-2e3a552b97eb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:40.995924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d8102-0ba7-4c11-a4e4-2e3a552b97eb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:40.996383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d8102-0ba7-4c11-a4e4-2e3a552b97eb None admin] PATH_INFO: `/v3/users/278121485902401ba4ae5ce91b57ec2f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:41.012894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d8102-0ba7-4c11-a4e4-2e3a552b97eb None admin] RBAC: Authorizing `identity:get_user(user_id=278121485902401ba4ae5ce91b57ec2f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:41.014743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d8102-0ba7-4c11-a4e4-2e3a552b97eb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:41.016832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 109/217] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:40 2022] GET /identity/v3/users/278121485902401ba4ae5ce91b57ec2f => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:41.022795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870b495-5d9c-4bf5-ace0-039285a70133 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:41.103308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:41.134734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870b495-5d9c-4bf5-ace0-039285a70133 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:41.171769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e84c6-1c5e-43fe-8e2f-14136e28226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:41.172525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e84c6-1c5e-43fe-8e2f-14136e28226c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:41.172810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e84c6-1c5e-43fe-8e2f-14136e28226c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:41.173088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e84c6-1c5e-43fe-8e2f-14136e28226c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:41.251412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e84c6-1c5e-43fe-8e2f-14136e28226c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:41.252884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7e84c6-1c5e-43fe-8e2f-14136e28226c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:41.332103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:41.366840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 109/218] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:41.372137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d8102-0ba7-4c11-a4e4-2e3a552b97eb None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:41.504093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d8102-0ba7-4c11-a4e4-2e3a552b97eb None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:41.507668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacdfdc-fc51-4e56-b001-fc6d58252b83 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:41.507958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacdfdc-fc51-4e56-b001-fc6d58252b83 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:41.508241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cacdfdc-fc51-4e56-b001-fc6d58252b83 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:41.509654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 110/219] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:41.523473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c300c3-b2af-4daf-bbb4-4fc32f753242 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:41.523758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c300c3-b2af-4daf-bbb4-4fc32f753242 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:41.524038 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c300c3-b2af-4daf-bbb4-4fc32f753242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:41.536002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:41.545368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe0a0f-6a01-4930-8826-4d23252a7f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:41.549053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe0a0f-6a01-4930-8826-4d23252a7f71 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:41.549339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe0a0f-6a01-4930-8826-4d23252a7f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:41.549612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe0a0f-6a01-4930-8826-4d23252a7f71 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:41.550995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-69c300c3-b2af-4daf-bbb4-4fc32f753242 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:41.552200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-69c300c3-b2af-4daf-bbb4-4fc32f753242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:41.554933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe0a0f-6a01-4930-8826-4d23252a7f71 None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:41.556392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fe0a0f-6a01-4930-8826-4d23252a7f71 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:41.565077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 110/220] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:41 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:41.565919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe0a0f-6a01-4930-8826-4d23252a7f71 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:41.679278 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 111/221] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:41.702880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c300c3-b2af-4daf-bbb4-4fc32f753242 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:41.759686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe0a0f-6a01-4930-8826-4d23252a7f71 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:41.812554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195dd8f5-0125-44d5-8dba-adf672e36f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:41.813221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dd8f5-0125-44d5-8dba-adf672e36f60 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:41.813505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dd8f5-0125-44d5-8dba-adf672e36f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:41.813782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195dd8f5-0125-44d5-8dba-adf672e36f60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:41.814651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195dd8f5-0125-44d5-8dba-adf672e36f60 None admin] RBAC: Adding query filter params (role.id=e1ed407e2d924fac8012250bb2ecf94f, scope.project.id=65124292b1924cc889516053c00bda74, user.id=278121485902401ba4ae5ce91b57ec2f) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:41.814965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195dd8f5-0125-44d5-8dba-adf672e36f60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:41.816407 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195dd8f5-0125-44d5-8dba-adf672e36f60 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:41.832551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 111/222] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:41 2022] GET /identity/v3/role_assignments?user.id=278121485902401ba4ae5ce91b57ec2f&scope.project.id=65124292b1924cc889516053c00bda74&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 259 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:41.885300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c300c3-b2af-4daf-bbb4-4fc32f753242 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:41.890432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:41.943199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d4745-e660-464c-9aa0-df7f073759f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:41.944002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d4745-e660-464c-9aa0-df7f073759f1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:41.944286 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d4745-e660-464c-9aa0-df7f073759f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:41.944614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2d4745-e660-464c-9aa0-df7f073759f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:41.962281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2d4745-e660-464c-9aa0-df7f073759f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1c2d4745-e660-464c-9aa0-df7f073759f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:41.970906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:41.971822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:41.972592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 112/223] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:51:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:41.979495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195dd8f5-0125-44d5-8dba-adf672e36f60 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:42.033488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:42.109730 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195dd8f5-0125-44d5-8dba-adf672e36f60 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:42.154151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fc413f-7122-476e-b1b2-363412146487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:42.154871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc413f-7122-476e-b1b2-363412146487 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:42.155163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc413f-7122-476e-b1b2-363412146487 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:42.155441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fc413f-7122-476e-b1b2-363412146487 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:42.156192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc413f-7122-476e-b1b2-363412146487 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:42.156502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc413f-7122-476e-b1b2-363412146487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:42.157873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fc413f-7122-476e-b1b2-363412146487 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:42.166572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 112/224] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:42.172645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d4745-e660-464c-9aa0-df7f073759f1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:42.222607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd70a4-7626-4ce1-acd4-f842231c03e2 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:42.222904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd70a4-7626-4ce1-acd4-f842231c03e2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:42.223085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd70a4-7626-4ce1-acd4-f842231c03e2 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:42.224633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 113/225] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:51:42.229375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12e197-621d-44e1-84de-0c74b02d736d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:42.229664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12e197-621d-44e1-84de-0c74b02d736d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:42.229941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b12e197-621d-44e1-84de-0c74b02d736d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:42.250459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5b12e197-621d-44e1-84de-0c74b02d736d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:42.251660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5b12e197-621d-44e1-84de-0c74b02d736d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:42.335870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2d4745-e660-464c-9aa0-df7f073759f1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:42.366681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 114/226] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:42.391137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12e197-621d-44e1-84de-0c74b02d736d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:42.400358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d44aa-cb71-4149-ba5d-4dced8a2204b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:42.401597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d44aa-cb71-4149-ba5d-4dced8a2204b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:42.402706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d44aa-cb71-4149-ba5d-4dced8a2204b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:42.403209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d44aa-cb71-4149-ba5d-4dced8a2204b None admin] PATH_INFO: `/v3/users/3e3cce5b2efb4a6eaaefb687f866d021` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:42.406920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d44aa-cb71-4149-ba5d-4dced8a2204b None admin] RBAC: Authorizing `identity:get_user(user_id=3e3cce5b2efb4a6eaaefb687f866d021)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:42.408742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d44aa-cb71-4149-ba5d-4dced8a2204b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:42.412898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 113/227] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:42 2022] GET /identity/v3/users/3e3cce5b2efb4a6eaaefb687f866d021 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:42.420357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d44aa-cb71-4149-ba5d-4dced8a2204b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:42.478554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:42.479925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:42.553698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b12e197-621d-44e1-84de-0c74b02d736d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:42.602737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cbcc3-d5cd-4e0d-85cd-2ed0bc9ca41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:42.604202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cbcc3-d5cd-4e0d-85cd-2ed0bc9ca41a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:42.604865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cbcc3-d5cd-4e0d-85cd-2ed0bc9ca41a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:42.606591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51cbcc3-d5cd-4e0d-85cd-2ed0bc9ca41a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:42.627580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51cbcc3-d5cd-4e0d-85cd-2ed0bc9ca41a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b51cbcc3-d5cd-4e0d-85cd-2ed0bc9ca41a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:42.635977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:42.636854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:42.637607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:42.647527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d44aa-cb71-4149-ba5d-4dced8a2204b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:42.654970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 115/228] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:51:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:42.661069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cbcc3-d5cd-4e0d-85cd-2ed0bc9ca41a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:42.713523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd099f7-357b-42df-94cb-796ce1a08f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:42.716557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd099f7-357b-42df-94cb-796ce1a08f67 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:42.717220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd099f7-357b-42df-94cb-796ce1a08f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:42.718299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd099f7-357b-42df-94cb-796ce1a08f67 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:42.719917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd099f7-357b-42df-94cb-796ce1a08f67 None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:42.722100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd099f7-357b-42df-94cb-796ce1a08f67 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:42.727528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 114/229] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:42 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:42.730682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd099f7-357b-42df-94cb-796ce1a08f67 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:42.858613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:42.863891 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51cbcc3-d5cd-4e0d-85cd-2ed0bc9ca41a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:42.901024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:42.911126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd099f7-357b-42df-94cb-796ce1a08f67 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:42.943641 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b89d5-5e6a-4044-84f5-9b980c229909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:42.944328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b89d5-5e6a-4044-84f5-9b980c229909 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:42.944659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b89d5-5e6a-4044-84f5-9b980c229909 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:42.944941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b89d5-5e6a-4044-84f5-9b980c229909 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:42.945706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b89d5-5e6a-4044-84f5-9b980c229909 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:42.946102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b89d5-5e6a-4044-84f5-9b980c229909 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:42.947513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b89d5-5e6a-4044-84f5-9b980c229909 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:42.960108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ca962-318b-40dd-aebf-bbc5d281efe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:42.961415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ca962-318b-40dd-aebf-bbc5d281efe7 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:42.961894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ca962-318b-40dd-aebf-bbc5d281efe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:42.962540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ca962-318b-40dd-aebf-bbc5d281efe7 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74/users/3e3cce5b2efb4a6eaaefb687f866d021/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:42.964776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ca962-318b-40dd-aebf-bbc5d281efe7 None admin] RBAC: Authorizing `identity:create_grant(project_id=65124292b1924cc889516053c00bda74, user_id=3e3cce5b2efb4a6eaaefb687f866d021, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:42.965089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 116/230] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:51:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:42.970871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ca962-318b-40dd-aebf-bbc5d281efe7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:42.972022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b89d5-5e6a-4044-84f5-9b980c229909 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:42.991083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 115/231] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:51:42 2022] PUT /identity/v3/projects/65124292b1924cc889516053c00bda74/users/3e3cce5b2efb4a6eaaefb687f866d021/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:51:43.106175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b89d5-5e6a-4044-84f5-9b980c229909 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:43.148857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be346f0-1bd2-40fb-9ac2-0808e7c7581f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:43.150147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be346f0-1bd2-40fb-9ac2-0808e7c7581f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:43.150651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be346f0-1bd2-40fb-9ac2-0808e7c7581f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:43.151159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be346f0-1bd2-40fb-9ac2-0808e7c7581f None admin] PATH_INFO: `/v3/users/3e3cce5b2efb4a6eaaefb687f866d021` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:43.152690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be346f0-1bd2-40fb-9ac2-0808e7c7581f None admin] RBAC: Authorizing `identity:get_user(user_id=3e3cce5b2efb4a6eaaefb687f866d021)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:43.154805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be346f0-1bd2-40fb-9ac2-0808e7c7581f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:43.156856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 117/232] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:42 2022] GET /identity/v3/users/3e3cce5b2efb4a6eaaefb687f866d021 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:43.162754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ca962-318b-40dd-aebf-bbc5d281efe7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:43.286682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ca962-318b-40dd-aebf-bbc5d281efe7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:43.323671 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59831aa-d5e8-4e3b-9bde-d6dfaa89e2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:43.324914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59831aa-d5e8-4e3b-9bde-d6dfaa89e2a2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:43.325394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59831aa-d5e8-4e3b-9bde-d6dfaa89e2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:43.325887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59831aa-d5e8-4e3b-9bde-d6dfaa89e2a2 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:43.327322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59831aa-d5e8-4e3b-9bde-d6dfaa89e2a2 None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:43.329097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59831aa-d5e8-4e3b-9bde-d6dfaa89e2a2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:43.331163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 116/233] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:43 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:43.337758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be346f0-1bd2-40fb-9ac2-0808e7c7581f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:43.375509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:43.445615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67b128-04d9-4892-a1d7-c66fb27082fc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:43.445962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67b128-04d9-4892-a1d7-c66fb27082fc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:43.446223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f67b128-04d9-4892-a1d7-c66fb27082fc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:43.447894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 117/234] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:43.455560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d651b6a-2987-418a-b70e-f898b82b2958 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:43.455850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d651b6a-2987-418a-b70e-f898b82b2958 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:43.456266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d651b6a-2987-418a-b70e-f898b82b2958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:43.485848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7d651b6a-2987-418a-b70e-f898b82b2958 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:43.488016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7d651b6a-2987-418a-b70e-f898b82b2958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:43.503955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be346f0-1bd2-40fb-9ac2-0808e7c7581f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:43.529204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:43.562421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de9659d-b272-4133-b1e6-5c59ffb32a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:43.568119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9659d-b272-4133-b1e6-5c59ffb32a9d None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:43.568615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9659d-b272-4133-b1e6-5c59ffb32a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:43.569088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de9659d-b272-4133-b1e6-5c59ffb32a9d None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74/users/3e3cce5b2efb4a6eaaefb687f866d021/roles/4f31bdaad0194bbd8fab39617059d14a` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:43.571615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9659d-b272-4133-b1e6-5c59ffb32a9d None admin] RBAC: Authorizing `identity:create_grant(project_id=65124292b1924cc889516053c00bda74, user_id=3e3cce5b2efb4a6eaaefb687f866d021, role_id=4f31bdaad0194bbd8fab39617059d14a)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:43.573356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de9659d-b272-4133-b1e6-5c59ffb32a9d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:43.596900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 118/235] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:51:43 2022] PUT /identity/v3/projects/65124292b1924cc889516053c00bda74/users/3e3cce5b2efb4a6eaaefb687f866d021/roles/4f31bdaad0194bbd8fab39617059d14a => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:51:43.612413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6bb24-9d7a-4169-be71-c3cb4ef8bbda None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:43.612976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6bb24-9d7a-4169-be71-c3cb4ef8bbda None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:43.613486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6bb24-9d7a-4169-be71-c3cb4ef8bbda None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:43.616828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 119/236] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:43.625561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55ff3a-58be-4f65-822d-d74bdba1f882 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:43.625870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55ff3a-58be-4f65-822d-d74bdba1f882 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:43.626157 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55ff3a-58be-4f65-822d-d74bdba1f882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:43.646459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 118/237] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:43.661229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-fb55ff3a-58be-4f65-822d-d74bdba1f882 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:43.662589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fb55ff3a-58be-4f65-822d-d74bdba1f882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:43.694369 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d651b6a-2987-418a-b70e-f898b82b2958 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:43.808065 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 120/238] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:43.833413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55ff3a-58be-4f65-822d-d74bdba1f882 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:43.858594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d651b6a-2987-418a-b70e-f898b82b2958 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:43.874452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:43.896702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83802f-e32b-4a69-b0d6-30485b863594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:43.897514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83802f-e32b-4a69-b0d6-30485b863594 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:43.897794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83802f-e32b-4a69-b0d6-30485b863594 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:43.898087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83802f-e32b-4a69-b0d6-30485b863594 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:43.912484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83802f-e32b-4a69-b0d6-30485b863594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9c83802f-e32b-4a69-b0d6-30485b863594 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:43.920478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:43.921358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:43.922596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 119/239] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:43.927696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83802f-e32b-4a69-b0d6-30485b863594 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:43.947639 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:43.967811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55ff3a-58be-4f65-822d-d74bdba1f882 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:44.002536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f2d18-4d7b-475c-8889-915c8cf5be48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:44.003339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f2d18-4d7b-475c-8889-915c8cf5be48 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:44.003624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f2d18-4d7b-475c-8889-915c8cf5be48 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:44.003900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14f2d18-4d7b-475c-8889-915c8cf5be48 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:44.005082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f2d18-4d7b-475c-8889-915c8cf5be48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:44.006566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14f2d18-4d7b-475c-8889-915c8cf5be48 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:44.008515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 121/240] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:44.014116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f2d18-4d7b-475c-8889-915c8cf5be48 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:44.110769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83802f-e32b-4a69-b0d6-30485b863594 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:44.184549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712359c-3916-42ee-9b09-f3a5795e9d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:44.185261 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712359c-3916-42ee-9b09-f3a5795e9d06 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:44.185542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712359c-3916-42ee-9b09-f3a5795e9d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:44.185816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b712359c-3916-42ee-9b09-f3a5795e9d06 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:44.186602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712359c-3916-42ee-9b09-f3a5795e9d06 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:44.186910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712359c-3916-42ee-9b09-f3a5795e9d06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:44.188288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b712359c-3916-42ee-9b09-f3a5795e9d06 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:44.197769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:44.205197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 120/241] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:44.210764 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712359c-3916-42ee-9b09-f3a5795e9d06 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:44.211266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14f2d18-4d7b-475c-8889-915c8cf5be48 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:44.273772 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d7c21-ce22-4d90-84ff-9a19d7f53a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:44.275442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d7c21-ce22-4d90-84ff-9a19d7f53a42 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:44.280896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d7c21-ce22-4d90-84ff-9a19d7f53a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:44.281403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d7c21-ce22-4d90-84ff-9a19d7f53a42 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:44.282383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d7c21-ce22-4d90-84ff-9a19d7f53a42 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:44.284079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d7c21-ce22-4d90-84ff-9a19d7f53a42 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:44.331295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 122/242] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:51:44 2022] POST /identity/v3/users => generated 293 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:51:44.344170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:44.375497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b712359c-3916-42ee-9b09-f3a5795e9d06 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:44.427008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425f2217-9f7e-429c-b93e-73a796fb4b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:44.428266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f2217-9f7e-429c-b93e-73a796fb4b33 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:44.428748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f2217-9f7e-429c-b93e-73a796fb4b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:44.429220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425f2217-9f7e-429c-b93e-73a796fb4b33 None admin] PATH_INFO: `/v3/users/da7288cfba674267a6df5e1028bd5936` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:44.450789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f2217-9f7e-429c-b93e-73a796fb4b33 None admin] RBAC: Authorizing `identity:get_user(user_id=da7288cfba674267a6df5e1028bd5936)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:44.452595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425f2217-9f7e-429c-b93e-73a796fb4b33 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:44.456049 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 121/243] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:44 2022] GET /identity/v3/users/da7288cfba674267a6df5e1028bd5936 => generated 297 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:44.461128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d7c21-ce22-4d90-84ff-9a19d7f53a42 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:44.582923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f30352-01d8-4635-b8a5-d62a53fe9f64 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:44.583538 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f30352-01d8-4635-b8a5-d62a53fe9f64 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:44.584069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f30352-01d8-4635-b8a5-d62a53fe9f64 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:44.587000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 122/244] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:44.592028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5819f-e258-4ab5-92a1-1917843ecae7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:44.592553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5819f-e258-4ab5-92a1-1917843ecae7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:44.593053 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5819f-e258-4ab5-92a1-1917843ecae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:44.604368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d7c21-ce22-4d90-84ff-9a19d7f53a42 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:44.612310 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2df5819f-e258-4ab5-92a1-1917843ecae7 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:44.613731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2df5819f-e258-4ab5-92a1-1917843ecae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:44.648666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfd8ee-8ea0-4fc8-85f4-d4a1966349bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:44.649268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfd8ee-8ea0-4fc8-85f4-d4a1966349bd None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:44.649557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfd8ee-8ea0-4fc8-85f4-d4a1966349bd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:44.649839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfd8ee-8ea0-4fc8-85f4-d4a1966349bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:44.669715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:44.698998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 123/245] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:44.721195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5819f-e258-4ab5-92a1-1917843ecae7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:44.732582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfd8ee-8ea0-4fc8-85f4-d4a1966349bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:44.734873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfd8ee-8ea0-4fc8-85f4-d4a1966349bd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:44.811186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:44.862783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5819f-e258-4ab5-92a1-1917843ecae7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:44.880130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 123/246] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:44.885727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfd8ee-8ea0-4fc8-85f4-d4a1966349bd None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:44.909050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90a03e-b13e-4a45-ae25-7eb8a0355d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:44.909944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90a03e-b13e-4a45-ae25-7eb8a0355d8d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:44.910368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90a03e-b13e-4a45-ae25-7eb8a0355d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:44.910747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90a03e-b13e-4a45-ae25-7eb8a0355d8d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:44.928870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90a03e-b13e-4a45-ae25-7eb8a0355d8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ea90a03e-b13e-4a45-ae25-7eb8a0355d8d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:44.937845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:44.938850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:44.939616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:44.940199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:44.942163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 124/247] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:44.948354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90a03e-b13e-4a45-ae25-7eb8a0355d8d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:45.001385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfd8ee-8ea0-4fc8-85f4-d4a1966349bd None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:45.040576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f99e0-961d-4a7f-a5b6-da928422de0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:45.041575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f99e0-961d-4a7f-a5b6-da928422de0e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:45.043614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f99e0-961d-4a7f-a5b6-da928422de0e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:45.043923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19f99e0-961d-4a7f-a5b6-da928422de0e None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:45.045300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f99e0-961d-4a7f-a5b6-da928422de0e None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:45.047360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19f99e0-961d-4a7f-a5b6-da928422de0e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:45.049478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 124/248] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:44 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:45.055127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f99e0-961d-4a7f-a5b6-da928422de0e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:45.061915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90a03e-b13e-4a45-ae25-7eb8a0355d8d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:45.102035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d2f75-b385-4f22-8e52-9d8191dfb040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:45.103250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2f75-b385-4f22-8e52-9d8191dfb040 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:45.103738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2f75-b385-4f22-8e52-9d8191dfb040 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:45.104379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d2f75-b385-4f22-8e52-9d8191dfb040 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:45.105974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d2f75-b385-4f22-8e52-9d8191dfb040 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:45.110843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:45.111696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d2f75-b385-4f22-8e52-9d8191dfb040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:45.113442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d2f75-b385-4f22-8e52-9d8191dfb040 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:45.127223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 125/249] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:45.134555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d2f75-b385-4f22-8e52-9d8191dfb040 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:45.184184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19f99e0-961d-4a7f-a5b6-da928422de0e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:45.199834 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:45.246943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822beaea-939f-402b-ab7a-824c78979156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:45.247862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822beaea-939f-402b-ab7a-824c78979156 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:45.248254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822beaea-939f-402b-ab7a-824c78979156 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:45.248881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822beaea-939f-402b-ab7a-824c78979156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:45.249823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822beaea-939f-402b-ab7a-824c78979156 None admin] RBAC: Adding query filter params (role.id=c2c12c04e72a4116bf5d8e58c2cedc7b, scope.project.id=65124292b1924cc889516053c00bda74, user.id=da7288cfba674267a6df5e1028bd5936) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:45.254663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822beaea-939f-402b-ab7a-824c78979156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:45.256236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822beaea-939f-402b-ab7a-824c78979156 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:45.257908 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d2f75-b385-4f22-8e52-9d8191dfb040 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:45.263569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 125/250] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:45 2022] GET /identity/v3/role_assignments?user.id=da7288cfba674267a6df5e1028bd5936&scope.project.id=65124292b1924cc889516053c00bda74&role.id=c2c12c04e72a4116bf5d8e58c2cedc7b => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:45.296883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf46a9-2362-448c-a941-381cb3e451de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:45.298014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf46a9-2362-448c-a941-381cb3e451de None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:45.298495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf46a9-2362-448c-a941-381cb3e451de None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:45.298930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cf46a9-2362-448c-a941-381cb3e451de None admin] PATH_INFO: `/v3/users/278121485902401ba4ae5ce91b57ec2f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:45.300291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf46a9-2362-448c-a941-381cb3e451de None admin] RBAC: Authorizing `identity:get_user(user_id=278121485902401ba4ae5ce91b57ec2f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:45.301892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cf46a9-2362-448c-a941-381cb3e451de None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:45.307782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 126/251] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:45 2022] GET /identity/v3/users/278121485902401ba4ae5ce91b57ec2f => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:45.310734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822beaea-939f-402b-ab7a-824c78979156 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:45.456841 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822beaea-939f-402b-ab7a-824c78979156 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:45.492840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894228d2-4670-4947-a5f0-743e4786a257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:45.493696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894228d2-4670-4947-a5f0-743e4786a257 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:45.493976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894228d2-4670-4947-a5f0-743e4786a257 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:45.494284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894228d2-4670-4947-a5f0-743e4786a257 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:45.495483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894228d2-4670-4947-a5f0-743e4786a257 None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:45.497074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894228d2-4670-4947-a5f0-743e4786a257 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:45.499299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 126/252] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:45 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:45.505097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf46a9-2362-448c-a941-381cb3e451de None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:45.535020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb804a8-c5ca-4111-9172-48d421ff3532 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:45.535347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb804a8-c5ca-4111-9172-48d421ff3532 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:45.535588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb804a8-c5ca-4111-9172-48d421ff3532 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:45.537250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 127/253] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:45.544313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b97a32-3209-41b5-88a2-acd9ab636a01 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:45.544602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b97a32-3209-41b5-88a2-acd9ab636a01 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:45.544908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b97a32-3209-41b5-88a2-acd9ab636a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:45.561576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f4b97a32-3209-41b5-88a2-acd9ab636a01 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:45.563036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f4b97a32-3209-41b5-88a2-acd9ab636a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:45.599473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:45.617364 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:45.673554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 128/254] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:45.692855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cf46a9-2362-448c-a941-381cb3e451de None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:45.719551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b97a32-3209-41b5-88a2-acd9ab636a01 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:45.741077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7b207-f01c-410d-bd9b-554775b573d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:45.741974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7b207-f01c-410d-bd9b-554775b573d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:45.742291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7b207-f01c-410d-bd9b-554775b573d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:45.742562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b7b207-f01c-410d-bd9b-554775b573d2 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74/users/278121485902401ba4ae5ce91b57ec2f/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:45.746980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7b207-f01c-410d-bd9b-554775b573d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=65124292b1924cc889516053c00bda74, user_id=278121485902401ba4ae5ce91b57ec2f, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:45.748696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b7b207-f01c-410d-bd9b-554775b573d2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:45.778882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 127/255] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:51:45 2022] PUT /identity/v3/projects/65124292b1924cc889516053c00bda74/users/278121485902401ba4ae5ce91b57ec2f/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:51:45.854293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b97a32-3209-41b5-88a2-acd9ab636a01 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:45.892369 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd93c8-a639-4669-9cb7-f9d727f03b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:45.893170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd93c8-a639-4669-9cb7-f9d727f03b60 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:45.893452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd93c8-a639-4669-9cb7-f9d727f03b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:45.893725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd93c8-a639-4669-9cb7-f9d727f03b60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:45.905301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd93c8-a639-4669-9cb7-f9d727f03b60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-6bcd93c8-a639-4669-9cb7-f9d727f03b60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:45.913144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:45.914075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:45.914897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 129/256] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:51:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:45.922353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7b207-f01c-410d-bd9b-554775b573d2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:46.051186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b7b207-f01c-410d-bd9b-554775b573d2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:46.069653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:46.096687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303febd2-6825-4fec-82f3-2b62557fdef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:46.097378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303febd2-6825-4fec-82f3-2b62557fdef6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:46.097662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303febd2-6825-4fec-82f3-2b62557fdef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:46.097946 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303febd2-6825-4fec-82f3-2b62557fdef6 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:46.098806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303febd2-6825-4fec-82f3-2b62557fdef6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:46.099118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303febd2-6825-4fec-82f3-2b62557fdef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:46.100519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303febd2-6825-4fec-82f3-2b62557fdef6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:46.114299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 128/257] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:46.122702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd93c8-a639-4669-9cb7-f9d727f03b60 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:46.169461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:46.244057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd93c8-a639-4669-9cb7-f9d727f03b60 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:46.280274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5cbf6-5723-4804-b1d7-ad8a040c9261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:46.281152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5cbf6-5723-4804-b1d7-ad8a040c9261 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:46.281437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5cbf6-5723-4804-b1d7-ad8a040c9261 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:46.281711 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5cbf6-5723-4804-b1d7-ad8a040c9261 None admin] PATH_INFO: `/v3/users/3e3cce5b2efb4a6eaaefb687f866d021` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:46.282958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5cbf6-5723-4804-b1d7-ad8a040c9261 None admin] RBAC: Authorizing `identity:get_user(user_id=3e3cce5b2efb4a6eaaefb687f866d021)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:46.284421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5cbf6-5723-4804-b1d7-ad8a040c9261 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:46.286421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 130/258] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:46 2022] GET /identity/v3/users/3e3cce5b2efb4a6eaaefb687f866d021 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:46.295173 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303febd2-6825-4fec-82f3-2b62557fdef6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:46.405113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e3e2f-c02f-427a-a362-5a3caa965861 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:46.405443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e3e2f-c02f-427a-a362-5a3caa965861 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:46.405682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e3e2f-c02f-427a-a362-5a3caa965861 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:46.407396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 131/259] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:46.412026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5b6f4-b16a-4013-9501-133d9ed8a381 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:46.412316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5b6f4-b16a-4013-9501-133d9ed8a381 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:46.412591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5b6f4-b16a-4013-9501-133d9ed8a381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:46.423275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303febd2-6825-4fec-82f3-2b62557fdef6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:46.451638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0ba5b6f4-b16a-4013-9501-133d9ed8a381 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:46.452863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0ba5b6f4-b16a-4013-9501-133d9ed8a381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:46.461331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71173e-2193-4592-9c44-3ff740212e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:46.462637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71173e-2193-4592-9c44-3ff740212e55 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:46.463315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71173e-2193-4592-9c44-3ff740212e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:46.464227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c71173e-2193-4592-9c44-3ff740212e55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:46.527861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:46.558370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 132/260] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:46.566069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71173e-2193-4592-9c44-3ff740212e55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:46.566069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c71173e-2193-4592-9c44-3ff740212e55 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:46.600998 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5b6f4-b16a-4013-9501-133d9ed8a381 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:46.651066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:46.669911 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 129/261] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:46.680907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71173e-2193-4592-9c44-3ff740212e55 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:46.749803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5b6f4-b16a-4013-9501-133d9ed8a381 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:46.794698 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:46.796290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bb6b3-474f-4995-8362-7009ad536aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:46.797327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bb6b3-474f-4995-8362-7009ad536aca None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:46.797900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bb6b3-474f-4995-8362-7009ad536aca None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:46.798480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bb6b3-474f-4995-8362-7009ad536aca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:46.819571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3bb6b3-474f-4995-8362-7009ad536aca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-8b3bb6b3-474f-4995-8362-7009ad536aca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:46.828183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:46.829066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:46.829831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:46.832384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 133/262] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:51:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:46.834821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c71173e-2193-4592-9c44-3ff740212e55 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:46.843048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bb6b3-474f-4995-8362-7009ad536aca None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:46.900731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c84d3-2b19-4808-99c8-b3eb900183ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:46.901669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c84d3-2b19-4808-99c8-b3eb900183ef None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:46.901957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c84d3-2b19-4808-99c8-b3eb900183ef None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:46.902742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0c84d3-2b19-4808-99c8-b3eb900183ef None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:46.906826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c84d3-2b19-4808-99c8-b3eb900183ef None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:46.908292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0c84d3-2b19-4808-99c8-b3eb900183ef None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:46.910412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 130/263] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:46 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:46.916100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c84d3-2b19-4808-99c8-b3eb900183ef None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:46.954212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:46.966755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3bb6b3-474f-4995-8362-7009ad536aca None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:47.010799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a144df8-6219-4575-8ed1-e896205aa1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:47.011778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144df8-6219-4575-8ed1-e896205aa1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:47.012280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144df8-6219-4575-8ed1-e896205aa1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:47.012695 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a144df8-6219-4575-8ed1-e896205aa1b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:47.013586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a144df8-6219-4575-8ed1-e896205aa1b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:47.014031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a144df8-6219-4575-8ed1-e896205aa1b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:47.015565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a144df8-6219-4575-8ed1-e896205aa1b5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:47.031311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 134/264] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:51:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:47.036860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a144df8-6219-4575-8ed1-e896205aa1b5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:47.156381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0c84d3-2b19-4808-99c8-b3eb900183ef None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:47.209044 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a144df8-6219-4575-8ed1-e896205aa1b5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:47.259289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08acc37-4e7e-4036-ac80-84ce262bd0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:47.259956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08acc37-4e7e-4036-ac80-84ce262bd0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:47.260238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08acc37-4e7e-4036-ac80-84ce262bd0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:47.260514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08acc37-4e7e-4036-ac80-84ce262bd0b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:47.261872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08acc37-4e7e-4036-ac80-84ce262bd0b7 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=65124292b1924cc889516053c00bda74, user.id=3e3cce5b2efb4a6eaaefb687f866d021) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:47.262223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08acc37-4e7e-4036-ac80-84ce262bd0b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:47.263669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08acc37-4e7e-4036-ac80-84ce262bd0b7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:47.264397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4381a-a663-4d32-a0f0-2941c99288ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:47.265529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4381a-a663-4d32-a0f0-2941c99288ba None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:47.265979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4381a-a663-4d32-a0f0-2941c99288ba None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:47.266429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4381a-a663-4d32-a0f0-2941c99288ba None admin] PATH_INFO: `/v3/users/3e3cce5b2efb4a6eaaefb687f866d021` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:47.270819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4381a-a663-4d32-a0f0-2941c99288ba None admin] RBAC: Authorizing `identity:get_user(user_id=3e3cce5b2efb4a6eaaefb687f866d021)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:47.272501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4381a-a663-4d32-a0f0-2941c99288ba None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:47.274339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 135/265] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:47 2022] GET /identity/v3/users/3e3cce5b2efb4a6eaaefb687f866d021 => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:47.277864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 131/266] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:46 2022] GET /identity/v3/role_assignments?user.id=3e3cce5b2efb4a6eaaefb687f866d021&scope.project.id=65124292b1924cc889516053c00bda74&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 608 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:47.280314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4381a-a663-4d32-a0f0-2941c99288ba None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:47.329492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:47.440752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4381a-a663-4d32-a0f0-2941c99288ba None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:47.522431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a3fa1-2e7f-430b-a244-eb98f35b40db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:47.523021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a3fa1-2e7f-430b-a244-eb98f35b40db None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:47.523312 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a3fa1-2e7f-430b-a244-eb98f35b40db None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:47.523592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a3fa1-2e7f-430b-a244-eb98f35b40db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:47.617571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a3fa1-2e7f-430b-a244-eb98f35b40db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:47.619100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a3fa1-2e7f-430b-a244-eb98f35b40db None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:47.658493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:47.750839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 136/267] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:47.756398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08acc37-4e7e-4036-ac80-84ce262bd0b7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:47.779117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ae090-51d3-41dc-81c0-8aa1dd7683df None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:47.779420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ae090-51d3-41dc-81c0-8aa1dd7683df None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:47.779597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ae090-51d3-41dc-81c0-8aa1dd7683df None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:47.781279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 137/268] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:47.782610 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:47.794648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a465e8-3b8e-4af7-b1ae-9a411a5b1270 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:47.794940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a465e8-3b8e-4af7-b1ae-9a411a5b1270 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:47.795192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a465e8-3b8e-4af7-b1ae-9a411a5b1270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:47.825182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-85a465e8-3b8e-4af7-b1ae-9a411a5b1270 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:47.826390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-85a465e8-3b8e-4af7-b1ae-9a411a5b1270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:47.899700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08acc37-4e7e-4036-ac80-84ce262bd0b7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:47.941081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32816a48-d41e-4ac8-8feb-8b55b71eaaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:47.942363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816a48-d41e-4ac8-8feb-8b55b71eaaad None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:47.942864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816a48-d41e-4ac8-8feb-8b55b71eaaad None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:47.943345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32816a48-d41e-4ac8-8feb-8b55b71eaaad None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:47.944908 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32816a48-d41e-4ac8-8feb-8b55b71eaaad None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:47.946718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32816a48-d41e-4ac8-8feb-8b55b71eaaad None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:47.948657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 132/269] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:47 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:47.949388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 138/270] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:47.954981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a465e8-3b8e-4af7-b1ae-9a411a5b1270 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:47.990504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32816a48-d41e-4ac8-8feb-8b55b71eaaad None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:48.045294 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:48.119839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a465e8-3b8e-4af7-b1ae-9a411a5b1270 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:48.131818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:48.162439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d4dfb-fafd-4fd9-b2cd-09b395ebc608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:48.164052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d4dfb-fafd-4fd9-b2cd-09b395ebc608 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.164727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d4dfb-fafd-4fd9-b2cd-09b395ebc608 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.165811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d4dfb-fafd-4fd9-b2cd-09b395ebc608 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.167018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d4dfb-fafd-4fd9-b2cd-09b395ebc608 None admin] RBAC: Adding query filter params (role.id=4f31bdaad0194bbd8fab39617059d14a, scope.project.id=65124292b1924cc889516053c00bda74, user.id=3e3cce5b2efb4a6eaaefb687f866d021) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:48.168603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d4dfb-fafd-4fd9-b2cd-09b395ebc608 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:48.170473 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d4dfb-fafd-4fd9-b2cd-09b395ebc608 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:48.175668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32816a48-d41e-4ac8-8feb-8b55b71eaaad None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:48.183088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 139/271] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:47 2022] GET /identity/v3/role_assignments?user.id=3e3cce5b2efb4a6eaaefb687f866d021&scope.project.id=65124292b1924cc889516053c00bda74&role.id=4f31bdaad0194bbd8fab39617059d14a => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:48.205205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ddabf-6549-4c49-b684-83fb17725687 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.205811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ddabf-6549-4c49-b684-83fb17725687 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.206328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433ddabf-6549-4c49-b684-83fb17725687 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.208025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 140/272] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:51:48.214115 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0657b9-8d56-4c48-87ce-e81f3c31f14c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.214775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0657b9-8d56-4c48-87ce-e81f3c31f14c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.215346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0657b9-8d56-4c48-87ce-e81f3c31f14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.224113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80726629-0e64-491d-84f2-54d28783b964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:48.224916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80726629-0e64-491d-84f2-54d28783b964 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.225202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80726629-0e64-491d-84f2-54d28783b964 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.225480 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80726629-0e64-491d-84f2-54d28783b964 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.234795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8e0657b9-8d56-4c48-87ce-e81f3c31f14c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:48.236427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8e0657b9-8d56-4c48-87ce-e81f3c31f14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:48.247334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80726629-0e64-491d-84f2-54d28783b964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-80726629-0e64-491d-84f2-54d28783b964 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:48.255405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:48.256263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:48.257032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:48.266297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 133/273] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:48.272383 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80726629-0e64-491d-84f2-54d28783b964 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:48.356861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 141/274] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:48.389620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0657b9-8d56-4c48-87ce-e81f3c31f14c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:48.438357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80726629-0e64-491d-84f2-54d28783b964 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:48.441771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:48.478772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a42d2a-11f9-4978-9aac-fc53b8e36066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:48.479801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a42d2a-11f9-4978-9aac-fc53b8e36066 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.479801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a42d2a-11f9-4978-9aac-fc53b8e36066 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.479975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a42d2a-11f9-4978-9aac-fc53b8e36066 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.480812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a42d2a-11f9-4978-9aac-fc53b8e36066 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:48.481124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a42d2a-11f9-4978-9aac-fc53b8e36066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:48.482633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a42d2a-11f9-4978-9aac-fc53b8e36066 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:48.488058 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:48.492870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 134/275] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:48.494180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2493436-b18c-48c0-9592-8cab83aeccc5 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.494819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2493436-b18c-48c0-9592-8cab83aeccc5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.495100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2493436-b18c-48c0-9592-8cab83aeccc5 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.497039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 135/276] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:48.498935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2493436-b18c-48c0-9592-8cab83aeccc5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:48.515915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0657b9-8d56-4c48-87ce-e81f3c31f14c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:48.555871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa7314b-8e98-43c8-874d-109f02cddfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:48.557069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7314b-8e98-43c8-874d-109f02cddfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.557552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7314b-8e98-43c8-874d-109f02cddfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.558032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa7314b-8e98-43c8-874d-109f02cddfd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.559996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7314b-8e98-43c8-874d-109f02cddfd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:48.561655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa7314b-8e98-43c8-874d-109f02cddfd0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:48.563576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 142/277] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:48.564611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78b3b-cb45-4883-9d09-5d4738fbd0ec None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.564983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78b3b-cb45-4883-9d09-5d4738fbd0ec None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.565328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb78b3b-cb45-4883-9d09-5d4738fbd0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.582650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3eb78b3b-cb45-4883-9d09-5d4738fbd0ec None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:48.584500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3eb78b3b-cb45-4883-9d09-5d4738fbd0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:48.629530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2493436-b18c-48c0-9592-8cab83aeccc5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:48.654727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 143/278] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:48.655783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb78b3b-cb45-4883-9d09-5d4738fbd0ec None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:48.676347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b8627-73a8-4cfc-a8e9-463871b6efc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:48.677232 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b8627-73a8-4cfc-a8e9-463871b6efc1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.677567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b8627-73a8-4cfc-a8e9-463871b6efc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.677846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54b8627-73a8-4cfc-a8e9-463871b6efc1 None admin] PATH_INFO: `/v3/users/d31fe712d8d646ab8d8a726b00357212` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.693923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b8627-73a8-4cfc-a8e9-463871b6efc1 None admin] RBAC: Authorizing `identity:get_user(user_id=d31fe712d8d646ab8d8a726b00357212)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:48.695419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54b8627-73a8-4cfc-a8e9-463871b6efc1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:48.697149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 136/279] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:48 2022] GET /identity/v3/users/d31fe712d8d646ab8d8a726b00357212 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:48.702928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b8627-73a8-4cfc-a8e9-463871b6efc1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:48.712585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:48.763297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:48.772443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb78b3b-cb45-4883-9d09-5d4738fbd0ec None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:48.805090 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54b8627-73a8-4cfc-a8e9-463871b6efc1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:48.807563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97e912-1083-451f-b082-ea1a0852129e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:48.808424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97e912-1083-451f-b082-ea1a0852129e None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.808846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97e912-1083-451f-b082-ea1a0852129e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.809260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97e912-1083-451f-b082-ea1a0852129e None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.810134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97e912-1083-451f-b082-ea1a0852129e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:48.811720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97e912-1083-451f-b082-ea1a0852129e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:48.833114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 144/280] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:51:48 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:51:48.834158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97e912-1083-451f-b082-ea1a0852129e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:48.855712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75163361-7318-4f25-b1ef-76360a137443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:48.856953 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75163361-7318-4f25-b1ef-76360a137443 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:48.857388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75163361-7318-4f25-b1ef-76360a137443 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:48.857809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75163361-7318-4f25-b1ef-76360a137443 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:48.871004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75163361-7318-4f25-b1ef-76360a137443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-75163361-7318-4f25-b1ef-76360a137443 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:48.880426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:48.881586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:48.882608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:48.885059 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 137/281] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:48.899116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75163361-7318-4f25-b1ef-76360a137443 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:48.966442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:48.969773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97e912-1083-451f-b082-ea1a0852129e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:49.013614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23bf2f9-8bfb-4005-8e30-6b440a9e88ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:49.014352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bf2f9-8bfb-4005-8e30-6b440a9e88ba None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:49.014637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bf2f9-8bfb-4005-8e30-6b440a9e88ba None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:49.014914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23bf2f9-8bfb-4005-8e30-6b440a9e88ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:49.015753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bf2f9-8bfb-4005-8e30-6b440a9e88ba None admin] RBAC: Adding query filter params (role.id=e1ed407e2d924fac8012250bb2ecf94f, user.id=d31fe712d8d646ab8d8a726b00357212, scope.system=all) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:49.016080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bf2f9-8bfb-4005-8e30-6b440a9e88ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:49.017677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23bf2f9-8bfb-4005-8e30-6b440a9e88ba None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:49.021516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75163361-7318-4f25-b1ef-76360a137443 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:49.025469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 145/282] 10.222.0.13 () {56 vars in 1308 bytes} [Sun Jun 26 02:51:48 2022] GET /identity/v3/role_assignments?user.id=d31fe712d8d646ab8d8a726b00357212&scope.system=all&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:49.026783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed801c-9a40-4b91-9a5d-bba3eae9f32e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:49.027105 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed801c-9a40-4b91-9a5d-bba3eae9f32e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:49.027379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed801c-9a40-4b91-9a5d-bba3eae9f32e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:49.028824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 146/283] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:49.034266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28fd48-6e33-4550-a317-d8fec6bb19ca None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:49.034556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28fd48-6e33-4550-a317-d8fec6bb19ca None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:49.034834 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc28fd48-6e33-4550-a317-d8fec6bb19ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:49.040145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:49.056896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cc28fd48-6e33-4550-a317-d8fec6bb19ca None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:49.060192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cc28fd48-6e33-4550-a317-d8fec6bb19ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:49.062261 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c1fb9-f033-4264-ac8d-772b46be3ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:49.062863 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c1fb9-f033-4264-ac8d-772b46be3ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:49.063153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c1fb9-f033-4264-ac8d-772b46be3ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:49.063437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224c1fb9-f033-4264-ac8d-772b46be3ee2 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:49.064214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c1fb9-f033-4264-ac8d-772b46be3ee2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:49.064532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c1fb9-f033-4264-ac8d-772b46be3ee2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:49.066687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224c1fb9-f033-4264-ac8d-772b46be3ee2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:49.079876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 138/284] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:49.085397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c1fb9-f033-4264-ac8d-772b46be3ee2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:49.158188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 147/285] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:49.191061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28fd48-6e33-4550-a317-d8fec6bb19ca None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:49.217854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224c1fb9-f033-4264-ac8d-772b46be3ee2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:49.271430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e833a5-06ee-4c70-9a67-cfaf176d5fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:49.272307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e833a5-06ee-4c70-9a67-cfaf176d5fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:49.272587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e833a5-06ee-4c70-9a67-cfaf176d5fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:49.272861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e833a5-06ee-4c70-9a67-cfaf176d5fb4 None admin] PATH_INFO: `/v3/users/da7288cfba674267a6df5e1028bd5936` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:49.274356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e833a5-06ee-4c70-9a67-cfaf176d5fb4 None admin] RBAC: Authorizing `identity:get_user(user_id=da7288cfba674267a6df5e1028bd5936)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:49.275520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e833a5-06ee-4c70-9a67-cfaf176d5fb4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:49.277506 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 139/286] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:49 2022] GET /identity/v3/users/da7288cfba674267a6df5e1028bd5936 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:49.283443 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e833a5-06ee-4c70-9a67-cfaf176d5fb4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:49.319986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:49.327855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc28fd48-6e33-4550-a317-d8fec6bb19ca None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:49.349570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:49.366159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d19c6-8766-4926-b7e8-71599aee59fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:49.367291 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d19c6-8766-4926-b7e8-71599aee59fa None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:49.367800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d19c6-8766-4926-b7e8-71599aee59fa None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:49.368271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442d19c6-8766-4926-b7e8-71599aee59fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:49.380305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442d19c6-8766-4926-b7e8-71599aee59fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-442d19c6-8766-4926-b7e8-71599aee59fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:49.388471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:49.389323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:49.390144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:49.392462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 148/287] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:49.398123 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d19c6-8766-4926-b7e8-71599aee59fa None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:49.406096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e833a5-06ee-4c70-9a67-cfaf176d5fb4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:49.442879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55276e86-20f4-4019-be70-f62536653166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:49.443560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55276e86-20f4-4019-be70-f62536653166 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:49.443839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55276e86-20f4-4019-be70-f62536653166 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:49.444113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55276e86-20f4-4019-be70-f62536653166 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:49.445300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55276e86-20f4-4019-be70-f62536653166 None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:49.447096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55276e86-20f4-4019-be70-f62536653166 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:49.454213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 140/288] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:49 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:49.463479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55276e86-20f4-4019-be70-f62536653166 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:49.578359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55276e86-20f4-4019-be70-f62536653166 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:49.610414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:49.621915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a9ab-f5a1-4aa4-b810-5b31e3dc1a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:49.622843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a9ab-f5a1-4aa4-b810-5b31e3dc1a54 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:49.623124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a9ab-f5a1-4aa4-b810-5b31e3dc1a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:49.623654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0718a9ab-f5a1-4aa4-b810-5b31e3dc1a54 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74/users/da7288cfba674267a6df5e1028bd5936/roles/c2c12c04e72a4116bf5d8e58c2cedc7b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:49.626681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718a9ab-f5a1-4aa4-b810-5b31e3dc1a54 None admin] RBAC: Authorizing `identity:create_grant(project_id=65124292b1924cc889516053c00bda74, user_id=da7288cfba674267a6df5e1028bd5936, role_id=c2c12c04e72a4116bf5d8e58c2cedc7b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:49.628126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0718a9ab-f5a1-4aa4-b810-5b31e3dc1a54 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:49.632966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442d19c6-8766-4926-b7e8-71599aee59fa None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:49.656513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 141/289] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:51:49 2022] PUT /identity/v3/projects/65124292b1924cc889516053c00bda74/users/da7288cfba674267a6df5e1028bd5936/roles/c2c12c04e72a4116bf5d8e58c2cedc7b => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:51:49.689017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9ed30-f2e7-413a-9124-c8c9912aeccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:49.689697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9ed30-f2e7-413a-9124-c8c9912aeccf None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:49.689982 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9ed30-f2e7-413a-9124-c8c9912aeccf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:49.690330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9ed30-f2e7-413a-9124-c8c9912aeccf None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:49.691168 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9ed30-f2e7-413a-9124-c8c9912aeccf None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:49.691475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9ed30-f2e7-413a-9124-c8c9912aeccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:49.692967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9ed30-f2e7-413a-9124-c8c9912aeccf None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:49.707871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 149/290] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:49.713725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a9ab-f5a1-4aa4-b810-5b31e3dc1a54 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:49.825131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0718a9ab-f5a1-4aa4-b810-5b31e3dc1a54 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:49.861768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66685ea-127f-48ca-8e50-4271e6318774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:49.862783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66685ea-127f-48ca-8e50-4271e6318774 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:49.863066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66685ea-127f-48ca-8e50-4271e6318774 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:49.863398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66685ea-127f-48ca-8e50-4271e6318774 None admin] PATH_INFO: `/v3/users/278121485902401ba4ae5ce91b57ec2f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:49.864711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66685ea-127f-48ca-8e50-4271e6318774 None admin] RBAC: Authorizing `identity:get_user(user_id=278121485902401ba4ae5ce91b57ec2f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:49.867939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66685ea-127f-48ca-8e50-4271e6318774 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:49.868429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 142/291] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:49 2022] GET /identity/v3/users/278121485902401ba4ae5ce91b57ec2f => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:49.876770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9ed30-f2e7-413a-9124-c8c9912aeccf None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:49.904940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:50.006533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9ed30-f2e7-413a-9124-c8c9912aeccf None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:50.046926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cacdf-0b9e-4c65-872a-cf23c5adfbdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:50.047919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cacdf-0b9e-4c65-872a-cf23c5adfbdc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:50.048406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cacdf-0b9e-4c65-872a-cf23c5adfbdc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:50.048873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cacdf-0b9e-4c65-872a-cf23c5adfbdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:50.128949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cacdf-0b9e-4c65-872a-cf23c5adfbdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:50.130755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cacdf-0b9e-4c65-872a-cf23c5adfbdc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:50.152217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:50.265331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 150/292] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:50.271207 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66685ea-127f-48ca-8e50-4271e6318774 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:50.358426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:50.424904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66685ea-127f-48ca-8e50-4271e6318774 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:50.457725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f286982e-3580-4984-ba9a-30f533bd4fd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:50.458624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286982e-3580-4984-ba9a-30f533bd4fd8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:50.458902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286982e-3580-4984-ba9a-30f533bd4fd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:50.459174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f286982e-3580-4984-ba9a-30f533bd4fd8 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:50.460358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286982e-3580-4984-ba9a-30f533bd4fd8 None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:50.461785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f286982e-3580-4984-ba9a-30f533bd4fd8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:50.463699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 143/293] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:50 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:50.469804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cacdf-0b9e-4c65-872a-cf23c5adfbdc None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:50.571872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cacdf-0b9e-4c65-872a-cf23c5adfbdc None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:50.597393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:50.622739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab30dcf1-f785-4c73-9351-1b7b02106aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:50.623661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30dcf1-f785-4c73-9351-1b7b02106aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:50.623951 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30dcf1-f785-4c73-9351-1b7b02106aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:50.624223 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab30dcf1-f785-4c73-9351-1b7b02106aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:50.625058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30dcf1-f785-4c73-9351-1b7b02106aa6 None admin] RBAC: Adding query filter params (role.id=e1ed407e2d924fac8012250bb2ecf94f, scope.project.id=65124292b1924cc889516053c00bda74, user.id=278121485902401ba4ae5ce91b57ec2f) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:50.625369 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30dcf1-f785-4c73-9351-1b7b02106aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:50.626819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab30dcf1-f785-4c73-9351-1b7b02106aa6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:50.638563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 151/294] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:50 2022] GET /identity/v3/role_assignments?user.id=278121485902401ba4ae5ce91b57ec2f&scope.project.id=65124292b1924cc889516053c00bda74&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:51.243789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59a76b-7940-409d-86c4-b4eb6da4b1a8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:51.244501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59a76b-7940-409d-86c4-b4eb6da4b1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:51.244999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db59a76b-7940-409d-86c4-b4eb6da4b1a8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:51.247461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 144/295] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:51.252939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec9e6c-2439-4204-b3a3-80f1544e7744 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:51.253781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec9e6c-2439-4204-b3a3-80f1544e7744 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:51.254297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec9e6c-2439-4204-b3a3-80f1544e7744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:51.273713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a5ec9e6c-2439-4204-b3a3-80f1544e7744 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:51.275253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a5ec9e6c-2439-4204-b3a3-80f1544e7744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:51.358093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 152/296] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:51.380275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59a76b-7940-409d-86c4-b4eb6da4b1a8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:51.488054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db59a76b-7940-409d-86c4-b4eb6da4b1a8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:51.497945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b178-b641-4ad2-8c6a-6969995c1ba2 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:51.498202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b178-b641-4ad2-8c6a-6969995c1ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:51.498736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f626b178-b641-4ad2-8c6a-6969995c1ba2 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:51.500144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 153/297] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:51.504944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defd3da-223f-4cfe-8b7b-5ead79c4ab95 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:51.505237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defd3da-223f-4cfe-8b7b-5ead79c4ab95 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:51.505516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0defd3da-223f-4cfe-8b7b-5ead79c4ab95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:51.510412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:51.525141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0defd3da-223f-4cfe-8b7b-5ead79c4ab95 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:51.526700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0defd3da-223f-4cfe-8b7b-5ead79c4ab95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:51.530242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf4973-c64d-4dec-abd2-5885f94a16dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:51.531164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf4973-c64d-4dec-abd2-5885f94a16dc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:51.531351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf4973-c64d-4dec-abd2-5885f94a16dc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:51.531580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf4973-c64d-4dec-abd2-5885f94a16dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:51.542197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf4973-c64d-4dec-abd2-5885f94a16dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-16bf4973-c64d-4dec-abd2-5885f94a16dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:51.550404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:51.551265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:51.552447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 145/298] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:51.557390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf4973-c64d-4dec-abd2-5885f94a16dc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:51.619880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 154/299] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:51.621290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ff81-cdf7-468c-8623-5c232b39169c None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:51.621582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ff81-cdf7-468c-8623-5c232b39169c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:51.621860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de95ff81-cdf7-468c-8623-5c232b39169c None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:51.623678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 155/300] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:51.628588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9134d-d05b-4e4b-963d-22d2acb60537 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:51.629036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9134d-d05b-4e4b-963d-22d2acb60537 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:51.629451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb9134d-d05b-4e4b-963d-22d2acb60537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:51.646387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7bb9134d-d05b-4e4b-963d-22d2acb60537 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:51.647558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7bb9134d-d05b-4e4b-963d-22d2acb60537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:51.687227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:51.687623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf4973-c64d-4dec-abd2-5885f94a16dc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:51.722381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49180ea0-2112-49af-9169-e30c3039c297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:51.723055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49180ea0-2112-49af-9169-e30c3039c297 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:51.723338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49180ea0-2112-49af-9169-e30c3039c297 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:51.723615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49180ea0-2112-49af-9169-e30c3039c297 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:51.724358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49180ea0-2112-49af-9169-e30c3039c297 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:51.724668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49180ea0-2112-49af-9169-e30c3039c297 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:51.726064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49180ea0-2112-49af-9169-e30c3039c297 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:51.726163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 156/301] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:51.732350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9134d-d05b-4e4b-963d-22d2acb60537 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:51.739274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 146/302] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:51.740318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49180ea0-2112-49af-9169-e30c3039c297 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:51.776718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:51.839195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb9134d-d05b-4e4b-963d-22d2acb60537 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:51.850670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49180ea0-2112-49af-9169-e30c3039c297 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:51.875479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37621b1e-6691-4898-9de6-bc0304c97b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:51.876365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37621b1e-6691-4898-9de6-bc0304c97b4c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:51.876723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37621b1e-6691-4898-9de6-bc0304c97b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:51.877072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37621b1e-6691-4898-9de6-bc0304c97b4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:51.878330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37621b1e-6691-4898-9de6-bc0304c97b4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:51.879786 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37621b1e-6691-4898-9de6-bc0304c97b4c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:51.881431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 157/303] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:51.882505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37621b1e-6691-4898-9de6-bc0304c97b4c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:51.895518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7200d3c-0928-426e-bde6-939ff9bdd442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:51.896394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7200d3c-0928-426e-bde6-939ff9bdd442 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:51.896677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7200d3c-0928-426e-bde6-939ff9bdd442 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:51.896954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7200d3c-0928-426e-bde6-939ff9bdd442 None admin] PATH_INFO: `/v3/users/d31fe712d8d646ab8d8a726b00357212` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:51.898170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7200d3c-0928-426e-bde6-939ff9bdd442 None admin] RBAC: Authorizing `identity:get_user(user_id=d31fe712d8d646ab8d8a726b00357212)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:51.899592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7200d3c-0928-426e-bde6-939ff9bdd442 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:51.901701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 147/304] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:51 2022] GET /identity/v3/users/d31fe712d8d646ab8d8a726b00357212 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:51.907516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7200d3c-0928-426e-bde6-939ff9bdd442 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:51.927034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:51.990767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37621b1e-6691-4898-9de6-bc0304c97b4c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:52.017244 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7200d3c-0928-426e-bde6-939ff9bdd442 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:52.019136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:52.029026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a10c4-d13a-429f-8c4e-2d60e7f451d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:52.029757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a10c4-d13a-429f-8c4e-2d60e7f451d6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.030067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a10c4-d13a-429f-8c4e-2d60e7f451d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.030348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a10c4-d13a-429f-8c4e-2d60e7f451d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.042120 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a10c4-d13a-429f-8c4e-2d60e7f451d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1e8a10c4-d13a-429f-8c4e-2d60e7f451d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:52.050286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:52.051150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:52.051992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 158/305] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:52.052639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a10c4-d13a-429f-8c4e-2d60e7f451d6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:52.053568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d88b7-ac6d-4210-b34a-4085f8d80479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:52.054256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d88b7-ac6d-4210-b34a-4085f8d80479 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.054696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d88b7-ac6d-4210-b34a-4085f8d80479 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.055170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5d88b7-ac6d-4210-b34a-4085f8d80479 None admin] PATH_INFO: `/v3/groups` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.056028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d88b7-ac6d-4210-b34a-4085f8d80479 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:52.057643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5d88b7-ac6d-4210-b34a-4085f8d80479 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:52.082081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 148/306] 10.222.0.13 () {60 vars in 1134 bytes} [Sun Jun 26 02:51:51 2022] POST /identity/v3/groups => generated 229 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:51:52.082081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d88b7-ac6d-4210-b34a-4085f8d80479 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:52.179509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:52.188433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a10c4-d13a-429f-8c4e-2d60e7f451d6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:52.223813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5d88b7-ac6d-4210-b34a-4085f8d80479 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:52.232089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfb5b38-2eaa-4eb2-ad25-c5ac9111b09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:52.232968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb5b38-2eaa-4eb2-ad25-c5ac9111b09e None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.233864 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb5b38-2eaa-4eb2-ad25-c5ac9111b09e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.234188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfb5b38-2eaa-4eb2-ad25-c5ac9111b09e None admin] PATH_INFO: `/v3/system/users/d31fe712d8d646ab8d8a726b00357212/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.236148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb5b38-2eaa-4eb2-ad25-c5ac9111b09e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d31fe712d8d646ab8d8a726b00357212, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:52.237874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfb5b38-2eaa-4eb2-ad25-c5ac9111b09e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:52.246963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 159/307] 10.222.0.13 () {58 vars in 1334 bytes} [Sun Jun 26 02:51:52 2022] PUT /identity/v3/system/users/d31fe712d8d646ab8d8a726b00357212/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:51:52.248734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de4a70-ffb9-4475-a8fa-48e04cfc9ebe None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.249032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de4a70-ffb9-4475-a8fa-48e04cfc9ebe None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.249319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08de4a70-ffb9-4475-a8fa-48e04cfc9ebe None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.250709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 160/308] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:52.256472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a4b85-c3fb-4042-9a7f-a5538cc936c5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.257086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a4b85-c3fb-4042-9a7f-a5538cc936c5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.257086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a4b85-c3fb-4042-9a7f-a5538cc936c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.267540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85ad10-f081-4aa5-af96-67d1f0a2e4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:52.268434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85ad10-f081-4aa5-af96-67d1f0a2e4d7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.269411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85ad10-f081-4aa5-af96-67d1f0a2e4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.269411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85ad10-f081-4aa5-af96-67d1f0a2e4d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.269944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85ad10-f081-4aa5-af96-67d1f0a2e4d7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:52.270285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85ad10-f081-4aa5-af96-67d1f0a2e4d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:52.271823 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85ad10-f081-4aa5-af96-67d1f0a2e4d7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:52.275880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0a5a4b85-c3fb-4042-9a7f-a5538cc936c5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:52.277657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0a5a4b85-c3fb-4042-9a7f-a5538cc936c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:52.286247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 149/309] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:52.289214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85ad10-f081-4aa5-af96-67d1f0a2e4d7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:52.322740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:52.373834 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 161/310] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:52.398786 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a4b85-c3fb-4042-9a7f-a5538cc936c5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:52.419259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85ad10-f081-4aa5-af96-67d1f0a2e4d7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:52.465012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b81a1-a0f3-4a71-9705-c6fb6c327a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:52.465903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b81a1-a0f3-4a71-9705-c6fb6c327a9e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.466208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b81a1-a0f3-4a71-9705-c6fb6c327a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.466482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276b81a1-a0f3-4a71-9705-c6fb6c327a9e None admin] PATH_INFO: `/v3/users/917b28b15e9c412a95d92b078b3c662f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.482122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b81a1-a0f3-4a71-9705-c6fb6c327a9e None admin] RBAC: Authorizing `identity:get_user(user_id=917b28b15e9c412a95d92b078b3c662f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:52.483602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276b81a1-a0f3-4a71-9705-c6fb6c327a9e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:52.485350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 150/311] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:52 2022] GET /identity/v3/users/917b28b15e9c412a95d92b078b3c662f => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:52.491067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b81a1-a0f3-4a71-9705-c6fb6c327a9e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:52.510251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:52.522270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a4b85-c3fb-4042-9a7f-a5538cc936c5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:52.562666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8664a-bc6c-42d0-878b-edb0956d719d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:52.563951 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8664a-bc6c-42d0-878b-edb0956d719d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.564491 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8664a-bc6c-42d0-878b-edb0956d719d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.564972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8664a-bc6c-42d0-878b-edb0956d719d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.576550 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:52.578627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d8664a-bc6c-42d0-878b-edb0956d719d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-14d8664a-bc6c-42d0-878b-edb0956d719d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:52.588957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:52.590773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:52.592194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:52.599290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276b81a1-a0f3-4a71-9705-c6fb6c327a9e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:52.599698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 162/312] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:52.601002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8664a-bc6c-42d0-878b-edb0956d719d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:52.635449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6e3af-8e48-471b-a093-6171ae2466e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:52.636116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6e3af-8e48-471b-a093-6171ae2466e5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.636926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6e3af-8e48-471b-a093-6171ae2466e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.637208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca6e3af-8e48-471b-a093-6171ae2466e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.638060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6e3af-8e48-471b-a093-6171ae2466e5 None admin] RBAC: Adding query filter params (role.id=c2c12c04e72a4116bf5d8e58c2cedc7b, user.id=917b28b15e9c412a95d92b078b3c662f, scope.system=all) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:52.638377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6e3af-8e48-471b-a093-6171ae2466e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:52.639866 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca6e3af-8e48-471b-a093-6171ae2466e5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:52.650162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 151/313] 10.222.0.13 () {56 vars in 1308 bytes} [Sun Jun 26 02:51:52 2022] GET /identity/v3/role_assignments?user.id=917b28b15e9c412a95d92b078b3c662f&scope.system=all&role.id=c2c12c04e72a4116bf5d8e58c2cedc7b => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:52.721382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d8664a-bc6c-42d0-878b-edb0956d719d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:52.760586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e649f-eaf8-4a69-a634-e3111960cfd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:52.762142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e649f-eaf8-4a69-a634-e3111960cfd8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.762580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e649f-eaf8-4a69-a634-e3111960cfd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.763004 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e649f-eaf8-4a69-a634-e3111960cfd8 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.763917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e649f-eaf8-4a69-a634-e3111960cfd8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:52.764404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e649f-eaf8-4a69-a634-e3111960cfd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:52.765966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8e649f-eaf8-4a69-a634-e3111960cfd8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:52.776257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 163/314] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:52.782171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6e3af-8e48-471b-a093-6171ae2466e5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:52.891634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca6e3af-8e48-471b-a093-6171ae2466e5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:52.926497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b8276-7fe0-4a01-8053-2a4186d70185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:52.927473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b8276-7fe0-4a01-8053-2a4186d70185 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:52.927759 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b8276-7fe0-4a01-8053-2a4186d70185 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:52.928036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72b8276-7fe0-4a01-8053-2a4186d70185 None admin] PATH_INFO: `/v3/users/da7288cfba674267a6df5e1028bd5936` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:52.929252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b8276-7fe0-4a01-8053-2a4186d70185 None admin] RBAC: Authorizing `identity:get_user(user_id=da7288cfba674267a6df5e1028bd5936)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:52.930755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72b8276-7fe0-4a01-8053-2a4186d70185 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:52.932728 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 152/315] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:52 2022] GET /identity/v3/users/da7288cfba674267a6df5e1028bd5936 => generated 297 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:52.938764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e649f-eaf8-4a69-a634-e3111960cfd8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:52.956638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:53.044912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8e649f-eaf8-4a69-a634-e3111960cfd8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:53.079715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b10283-361b-49ea-9fe2-21f704c01b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:53.080374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b10283-361b-49ea-9fe2-21f704c01b72 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:53.080720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b10283-361b-49ea-9fe2-21f704c01b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:53.081004 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b10283-361b-49ea-9fe2-21f704c01b72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:53.150436 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b10283-361b-49ea-9fe2-21f704c01b72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:53.151940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b10283-361b-49ea-9fe2-21f704c01b72 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:53.167015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:53.254419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 164/316] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:53.260055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b8276-7fe0-4a01-8053-2a4186d70185 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:53.277839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:53.367356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72b8276-7fe0-4a01-8053-2a4186d70185 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:53.404674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b249081-1ec8-41d1-8715-a90f9f297c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:53.405458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b249081-1ec8-41d1-8715-a90f9f297c32 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:53.405740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b249081-1ec8-41d1-8715-a90f9f297c32 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:53.406008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b249081-1ec8-41d1-8715-a90f9f297c32 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:53.407224 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b249081-1ec8-41d1-8715-a90f9f297c32 None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:53.408652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b249081-1ec8-41d1-8715-a90f9f297c32 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:53.410775 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 153/317] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:53 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:53.416447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b10283-361b-49ea-9fe2-21f704c01b72 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:53.526405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b10283-361b-49ea-9fe2-21f704c01b72 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:53.540726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:53.562252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d20dee-5f2f-49d2-a857-d2d1a09cd39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:53.562916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d20dee-5f2f-49d2-a857-d2d1a09cd39b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:53.563203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d20dee-5f2f-49d2-a857-d2d1a09cd39b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:53.563484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d20dee-5f2f-49d2-a857-d2d1a09cd39b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:53.564324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d20dee-5f2f-49d2-a857-d2d1a09cd39b None admin] RBAC: Adding query filter params (role.id=c2c12c04e72a4116bf5d8e58c2cedc7b, scope.project.id=65124292b1924cc889516053c00bda74, user.id=da7288cfba674267a6df5e1028bd5936) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:53.564644 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d20dee-5f2f-49d2-a857-d2d1a09cd39b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:53.566103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d20dee-5f2f-49d2-a857-d2d1a09cd39b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:53.572642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 165/318] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:51:53 2022] GET /identity/v3/role_assignments?user.id=da7288cfba674267a6df5e1028bd5936&scope.project.id=65124292b1924cc889516053c00bda74&role.id=c2c12c04e72a4116bf5d8e58c2cedc7b => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:54.228404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb6072-44f6-498e-bb17-0bfba7137f2e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:54.228912 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb6072-44f6-498e-bb17-0bfba7137f2e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:54.228988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bb6072-44f6-498e-bb17-0bfba7137f2e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:54.230955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 154/319] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:54.235921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47a9a7-55b7-4dea-aab3-35f2bfcd5deb None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:54.236213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47a9a7-55b7-4dea-aab3-35f2bfcd5deb None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:54.236490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47a9a7-55b7-4dea-aab3-35f2bfcd5deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:54.253301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4f47a9a7-55b7-4dea-aab3-35f2bfcd5deb None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:54.254512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4f47a9a7-55b7-4dea-aab3-35f2bfcd5deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:54.329534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 166/320] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:54.351965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb6072-44f6-498e-bb17-0bfba7137f2e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:54.434701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:54.449092 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216765-f626-4ea9-ad5b-1d40f680b94a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:54.449592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216765-f626-4ea9-ad5b-1d40f680b94a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:54.449681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae216765-f626-4ea9-ad5b-1d40f680b94a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:54.451076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 167/321] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:54.456063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd59f0b-8eef-4c23-bb28-07cc555c2e16 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:54.456349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd59f0b-8eef-4c23-bb28-07cc555c2e16 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:54.456789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd59f0b-8eef-4c23-bb28-07cc555c2e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:54.458966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bb6072-44f6-498e-bb17-0bfba7137f2e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:54.473803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5dd59f0b-8eef-4c23-bb28-07cc555c2e16 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:54.475020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5dd59f0b-8eef-4c23-bb28-07cc555c2e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:54.494933 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f6ad8-3bfd-4b00-8900-1b2556e3ef9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:54.495675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f6ad8-3bfd-4b00-8900-1b2556e3ef9c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:54.495954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f6ad8-3bfd-4b00-8900-1b2556e3ef9c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:54.496225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55f6ad8-3bfd-4b00-8900-1b2556e3ef9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:54.497795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f6ad8-3bfd-4b00-8900-1b2556e3ef9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:54.499549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55f6ad8-3bfd-4b00-8900-1b2556e3ef9c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:54.505534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 155/322] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:51:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:54.506451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f6ad8-3bfd-4b00-8900-1b2556e3ef9c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:54.550081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 168/323] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:54.571977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd59f0b-8eef-4c23-bb28-07cc555c2e16 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:54.613134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55f6ad8-3bfd-4b00-8900-1b2556e3ef9c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:54.616439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:54.649368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7843058-a237-4607-8b01-0ce773c5882f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:54.650035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7843058-a237-4607-8b01-0ce773c5882f None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:54.650341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7843058-a237-4607-8b01-0ce773c5882f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:54.650616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7843058-a237-4607-8b01-0ce773c5882f None admin] PATH_INFO: `/v3/groups` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:54.651299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7843058-a237-4607-8b01-0ce773c5882f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:54.653346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7843058-a237-4607-8b01-0ce773c5882f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:54.668529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 156/324] 10.222.0.13 () {60 vars in 1134 bytes} [Sun Jun 26 02:51:54 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:51:54.683605 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd59f0b-8eef-4c23-bb28-07cc555c2e16 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:54.722970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488a650-ca5b-4474-9b21-2b15296523a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:54.723858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488a650-ca5b-4474-9b21-2b15296523a2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:54.724187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488a650-ca5b-4474-9b21-2b15296523a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:54.724470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488a650-ca5b-4474-9b21-2b15296523a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:54.737306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488a650-ca5b-4474-9b21-2b15296523a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b488a650-ca5b-4474-9b21-2b15296523a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:54.745451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:54.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:54.747800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 169/325] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:54.753384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7843058-a237-4607-8b01-0ce773c5882f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:54.765758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d1f02-5cb2-40e6-9878-2b1a22b11f77 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:54.766114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d1f02-5cb2-40e6-9878-2b1a22b11f77 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:54.766358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d1f02-5cb2-40e6-9878-2b1a22b11f77 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:54.767384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:54.767784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 170/326] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:51:54.773297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a76ca-9de9-466f-b785-85031b1727e1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:54.773591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a76ca-9de9-466f-b785-85031b1727e1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:54.773860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a76ca-9de9-466f-b785-85031b1727e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:54.791149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ba7a76ca-9de9-466f-b785-85031b1727e1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:54.792990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ba7a76ca-9de9-466f-b785-85031b1727e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:54.872469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 171/327] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:54.881947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7843058-a237-4607-8b01-0ce773c5882f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:54.894030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a76ca-9de9-466f-b785-85031b1727e1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:54.923621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5476e-0a9e-4036-9988-b06f831128e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:54.924541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5476e-0a9e-4036-9988-b06f831128e9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:54.924975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5476e-0a9e-4036-9988-b06f831128e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:54.925420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a5476e-0a9e-4036-9988-b06f831128e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:54.926359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5476e-0a9e-4036-9988-b06f831128e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:54.926889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5476e-0a9e-4036-9988-b06f831128e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:54.928562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a5476e-0a9e-4036-9988-b06f831128e9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:54.928929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:54.942898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 157/328] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:54.949502 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5476e-0a9e-4036-9988-b06f831128e9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:55.029882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a76ca-9de9-466f-b785-85031b1727e1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:55.040596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:55.072177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c8702-bf39-4de7-9a17-02922a7abcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:55.072976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c8702-bf39-4de7-9a17-02922a7abcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:55.073261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c8702-bf39-4de7-9a17-02922a7abcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:55.073538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c8702-bf39-4de7-9a17-02922a7abcb7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:55.076155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a5476e-0a9e-4036-9988-b06f831128e9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:55.091533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48c8702-bf39-4de7-9a17-02922a7abcb7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-d48c8702-bf39-4de7-9a17-02922a7abcb7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:55.100049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:55.100955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:55.102340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 172/329] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:51:55.110191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c8702-bf39-4de7-9a17-02922a7abcb7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:55.119002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafd35f-8f85-420b-9513-e488fff91fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:55.120175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafd35f-8f85-420b-9513-e488fff91fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:55.120727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafd35f-8f85-420b-9513-e488fff91fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:55.121189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdafd35f-8f85-420b-9513-e488fff91fb2 None admin] PATH_INFO: `/v3/users/d31fe712d8d646ab8d8a726b00357212` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:55.123108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafd35f-8f85-420b-9513-e488fff91fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=d31fe712d8d646ab8d8a726b00357212)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:55.125154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdafd35f-8f85-420b-9513-e488fff91fb2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:55.127227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 158/330] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:54 2022] GET /identity/v3/users/d31fe712d8d646ab8d8a726b00357212 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:55.133585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafd35f-8f85-420b-9513-e488fff91fb2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:55.225451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:55.243029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c8702-bf39-4de7-9a17-02922a7abcb7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:55.256289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdafd35f-8f85-420b-9513-e488fff91fb2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:55.287610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f054b9-8563-44d8-8abf-6a58d086b631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:55.288352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f054b9-8563-44d8-8abf-6a58d086b631 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:55.288642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f054b9-8563-44d8-8abf-6a58d086b631 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:55.288920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f054b9-8563-44d8-8abf-6a58d086b631 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:55.289684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f054b9-8563-44d8-8abf-6a58d086b631 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:55.290170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f054b9-8563-44d8-8abf-6a58d086b631 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:55.292537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f054b9-8563-44d8-8abf-6a58d086b631 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:55.306095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57220bf0-bd87-4437-a448-b6aef5144725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:55.306095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57220bf0-bd87-4437-a448-b6aef5144725 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:55.306095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57220bf0-bd87-4437-a448-b6aef5144725 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:55.306095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57220bf0-bd87-4437-a448-b6aef5144725 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:55.306485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57220bf0-bd87-4437-a448-b6aef5144725 None admin] RBAC: Adding query filter params (role.id=e1ed407e2d924fac8012250bb2ecf94f, user.id=d31fe712d8d646ab8d8a726b00357212, scope.system=all) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:55.306485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57220bf0-bd87-4437-a448-b6aef5144725 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:55.306785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 173/331] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:55.308320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57220bf0-bd87-4437-a448-b6aef5144725 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:55.309412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f054b9-8563-44d8-8abf-6a58d086b631 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:55.317785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:55.330799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 159/332] 10.222.0.13 () {56 vars in 1308 bytes} [Sun Jun 26 02:51:55 2022] GET /identity/v3/role_assignments?user.id=d31fe712d8d646ab8d8a726b00357212&scope.system=all&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:55.427399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f054b9-8563-44d8-8abf-6a58d086b631 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:55.469623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bd8cba-5d77-479b-8800-957d988ac608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:55.471436 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd8cba-5d77-479b-8800-957d988ac608 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:55.471962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd8cba-5d77-479b-8800-957d988ac608 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:55.472444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bd8cba-5d77-479b-8800-957d988ac608 None admin] PATH_INFO: `/v3/users/917b28b15e9c412a95d92b078b3c662f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:55.474799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd8cba-5d77-479b-8800-957d988ac608 None admin] RBAC: Authorizing `identity:get_user(user_id=917b28b15e9c412a95d92b078b3c662f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:55.476500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bd8cba-5d77-479b-8800-957d988ac608 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:55.478880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 174/333] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:51:55 2022] GET /identity/v3/users/917b28b15e9c412a95d92b078b3c662f => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:55.485141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57220bf0-bd87-4437-a448-b6aef5144725 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:55.607570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57220bf0-bd87-4437-a448-b6aef5144725 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:55.643858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea26e611-666c-405c-811d-6292731ef95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:55.644727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26e611-666c-405c-811d-6292731ef95f None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:55.645009 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26e611-666c-405c-811d-6292731ef95f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:55.645286 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea26e611-666c-405c-811d-6292731ef95f None admin] PATH_INFO: `/v3/system/users/917b28b15e9c412a95d92b078b3c662f/roles/c2c12c04e72a4116bf5d8e58c2cedc7b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:55.646809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26e611-666c-405c-811d-6292731ef95f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=917b28b15e9c412a95d92b078b3c662f, role_id=c2c12c04e72a4116bf5d8e58c2cedc7b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:55.648198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea26e611-666c-405c-811d-6292731ef95f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:55.660614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 160/334] 10.222.0.13 () {58 vars in 1334 bytes} [Sun Jun 26 02:51:55 2022] PUT /identity/v3/system/users/917b28b15e9c412a95d92b078b3c662f/roles/c2c12c04e72a4116bf5d8e58c2cedc7b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:51:57.027970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635525c-daad-44d4-9e41-eaee2731f668 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:57.028965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635525c-daad-44d4-9e41-eaee2731f668 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:57.029502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1635525c-daad-44d4-9e41-eaee2731f668 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:57.031195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 175/335] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:57.036729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368df25-0b6b-4255-8f23-96f6ef17faf9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:57.037160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368df25-0b6b-4255-8f23-96f6ef17faf9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:57.037339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0368df25-0b6b-4255-8f23-96f6ef17faf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:57.063088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0368df25-0b6b-4255-8f23-96f6ef17faf9 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:57.064326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0368df25-0b6b-4255-8f23-96f6ef17faf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:57.134835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:57.149596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 161/336] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:57.185741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635525c-daad-44d4-9e41-eaee2731f668 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:57.207144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:57.323030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1635525c-daad-44d4-9e41-eaee2731f668 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:57.368072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c84e1-b978-4891-bcb8-b1eaaf5d2fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:57.368869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c84e1-b978-4891-bcb8-b1eaaf5d2fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:57.369153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c84e1-b978-4891-bcb8-b1eaaf5d2fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:57.369439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c84e1-b978-4891-bcb8-b1eaaf5d2fe8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:57.385939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c84e1-b978-4891-bcb8-b1eaaf5d2fe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1b9c84e1-b978-4891-bcb8-b1eaaf5d2fe8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:57.394210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:57.395079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:57.396015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 176/337] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:57.401099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368df25-0b6b-4255-8f23-96f6ef17faf9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:57.615074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0368df25-0b6b-4255-8f23-96f6ef17faf9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:57.661995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a2456-a306-40bc-ab4e-472d3f630f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:57.662620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a2456-a306-40bc-ab4e-472d3f630f09 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:57.662907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a2456-a306-40bc-ab4e-472d3f630f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:57.663185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a2456-a306-40bc-ab4e-472d3f630f09 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:57.663943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a2456-a306-40bc-ab4e-472d3f630f09 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:57.664255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a2456-a306-40bc-ab4e-472d3f630f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:57.665630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a2456-a306-40bc-ab4e-472d3f630f09 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:57.679046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 162/338] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:51:57.684693 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c84e1-b978-4891-bcb8-b1eaaf5d2fe8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:57.697539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1c288-ed74-4686-a7f4-12f82e8e466b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:57.697823 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1c288-ed74-4686-a7f4-12f82e8e466b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:57.698156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1c288-ed74-4686-a7f4-12f82e8e466b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:57.700868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 163/339] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:57.709639 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9bab1-d56c-4d99-96ee-c09fd47ee0c4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:57.709930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9bab1-d56c-4d99-96ee-c09fd47ee0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:57.710238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9bab1-d56c-4d99-96ee-c09fd47ee0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:57.757299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-68c9bab1-d56c-4d99-96ee-c09fd47ee0c4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:57.758537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-68c9bab1-d56c-4d99-96ee-c09fd47ee0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:57.831416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:57.844624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:57.866227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c84e1-b978-4891-bcb8-b1eaaf5d2fe8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:57.876203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 164/340] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 26 02:51:57.876203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193d4ef-19a2-4c3b-ab4e-904bebdb45cc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:57.876203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193d4ef-19a2-4c3b-ab4e-904bebdb45cc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:57.876203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c193d4ef-19a2-4c3b-ab4e-904bebdb45cc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:57.877048 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 165/341] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:57.892076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29799cc-db0a-43f2-b96c-ca9f87de96ce None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:57.892411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29799cc-db0a-43f2-b96c-ca9f87de96ce None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:57.892640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29799cc-db0a-43f2-b96c-ca9f87de96ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:57.914621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b29799cc-db0a-43f2-b96c-ca9f87de96ce None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:57.916028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b29799cc-db0a-43f2-b96c-ca9f87de96ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:57.921052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6534f-3269-4104-bfe8-02e1730a4767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:57.921766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6534f-3269-4104-bfe8-02e1730a4767 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:57.922022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6534f-3269-4104-bfe8-02e1730a4767 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:57.922328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a6534f-3269-4104-bfe8-02e1730a4767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:58.013570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 166/342] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:51:58.014533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29799cc-db0a-43f2-b96c-ca9f87de96ce None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:58.024292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6534f-3269-4104-bfe8-02e1730a4767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:58.026226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a6534f-3269-4104-bfe8-02e1730a4767 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:58.160117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:58.167448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29799cc-db0a-43f2-b96c-ca9f87de96ce None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:58.193918 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 177/343] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:58.201844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6534f-3269-4104-bfe8-02e1730a4767 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:58.202653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:58.238285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59111029-104e-4cd0-8f4e-04177595d74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:58.239112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59111029-104e-4cd0-8f4e-04177595d74e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:58.239394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59111029-104e-4cd0-8f4e-04177595d74e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:58.239668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59111029-104e-4cd0-8f4e-04177595d74e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:58.251098 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59111029-104e-4cd0-8f4e-04177595d74e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-59111029-104e-4cd0-8f4e-04177595d74e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:58.259371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:58.260231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:58.260995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 167/344] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:51:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:58.262208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59111029-104e-4cd0-8f4e-04177595d74e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:58.351123 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a6534f-3269-4104-bfe8-02e1730a4767 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:58.387178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59111029-104e-4cd0-8f4e-04177595d74e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:58.418845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d926306-fe05-4eea-898a-dafc9b71f8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:58.419583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d926306-fe05-4eea-898a-dafc9b71f8d7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:58.419873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d926306-fe05-4eea-898a-dafc9b71f8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:58.420151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d926306-fe05-4eea-898a-dafc9b71f8d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:58.424614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d620012-ddbf-4692-9513-758819b35ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:58.425484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d620012-ddbf-4692-9513-758819b35ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:58.425764 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d620012-ddbf-4692-9513-758819b35ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:58.426054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d620012-ddbf-4692-9513-758819b35ba3 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:58.427363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d620012-ddbf-4692-9513-758819b35ba3 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:58.429003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d620012-ddbf-4692-9513-758819b35ba3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:58.430878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 168/345] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:58 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:58.431648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d620012-ddbf-4692-9513-758819b35ba3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:58.437442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d926306-fe05-4eea-898a-dafc9b71f8d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-4d926306-fe05-4eea-898a-dafc9b71f8d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:58.457554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:58.458535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:51:58.459405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 178/346] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:58.460049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d926306-fe05-4eea-898a-dafc9b71f8d7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:58.486387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:58.533880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:58.612047 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d620012-ddbf-4692-9513-758819b35ba3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:58.640378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d926306-fe05-4eea-898a-dafc9b71f8d7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:58.658210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294974da-617b-4539-bd1a-02efcfde3cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:58.658903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294974da-617b-4539-bd1a-02efcfde3cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:58.659188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294974da-617b-4539-bd1a-02efcfde3cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:58.659467 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294974da-617b-4539-bd1a-02efcfde3cb7 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:58.660226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294974da-617b-4539-bd1a-02efcfde3cb7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:58.660538 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294974da-617b-4539-bd1a-02efcfde3cb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:58.661914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294974da-617b-4539-bd1a-02efcfde3cb7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:58.673351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 169/347] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:51:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:58.674259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294974da-617b-4539-bd1a-02efcfde3cb7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:58.682809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10f179f-742f-4a27-b3b5-575d1f207d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:58.683741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f179f-742f-4a27-b3b5-575d1f207d0c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:58.684030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f179f-742f-4a27-b3b5-575d1f207d0c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:58.684312 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10f179f-742f-4a27-b3b5-575d1f207d0c None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:58.690971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f179f-742f-4a27-b3b5-575d1f207d0c None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:58.692474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10f179f-742f-4a27-b3b5-575d1f207d0c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:58.695123 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 179/348] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:51:58 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:58.696385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37f970-1774-4933-b05d-329fafa1c461 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:58.696674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37f970-1774-4933-b05d-329fafa1c461 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:58.696956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e37f970-1774-4933-b05d-329fafa1c461 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:58.698521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 180/349] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:58.699285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37f970-1774-4933-b05d-329fafa1c461 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:58.793312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294974da-617b-4539-bd1a-02efcfde3cb7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:58.795771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:58.822209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e37f970-1774-4933-b05d-329fafa1c461 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:58.825085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:58.835893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c53f22-95ca-47ef-93c1-e8ffd61e0bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:58.836854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c53f22-95ca-47ef-93c1-e8ffd61e0bef None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:58.837143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c53f22-95ca-47ef-93c1-e8ffd61e0bef None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:58.837421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c53f22-95ca-47ef-93c1-e8ffd61e0bef None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:58.838204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c53f22-95ca-47ef-93c1-e8ffd61e0bef None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:58.838720 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c53f22-95ca-47ef-93c1-e8ffd61e0bef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:58.841266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c53f22-95ca-47ef-93c1-e8ffd61e0bef None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:58.852136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 170/350] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:58.853017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c53f22-95ca-47ef-93c1-e8ffd61e0bef None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:58.879585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525529b-2ed3-4cd7-81dc-0cf810e3a932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:58.880336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525529b-2ed3-4cd7-81dc-0cf810e3a932 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:58.880726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525529b-2ed3-4cd7-81dc-0cf810e3a932 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:58.881073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525529b-2ed3-4cd7-81dc-0cf810e3a932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:58.956937 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525529b-2ed3-4cd7-81dc-0cf810e3a932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:58.958705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9525529b-2ed3-4cd7-81dc-0cf810e3a932 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:58.981284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c53f22-95ca-47ef-93c1-e8ffd61e0bef None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:59.019967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ab01f-c176-4fad-ac78-fa949ce65fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:59.020876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ab01f-c176-4fad-ac78-fa949ce65fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.021309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ab01f-c176-4fad-ac78-fa949ce65fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.021736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ab01f-c176-4fad-ac78-fa949ce65fb9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.023788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ab01f-c176-4fad-ac78-fa949ce65fb9 None admin] RBAC: Adding query filter params (group.id=af2ff3e5a07c4cf98efc807633fe9b8f, role.id=e1ed407e2d924fac8012250bb2ecf94f, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:51:59.024270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ab01f-c176-4fad-ac78-fa949ce65fb9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:59.025923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ab01f-c176-4fad-ac78-fa949ce65fb9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:59.032603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 171/351] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Jun 26 02:51:58 2022] GET /identity/v3/role_assignments?group.id=af2ff3e5a07c4cf98efc807633fe9b8f&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:59.034063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61018c-d044-4539-a079-25676f3dbe73 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.034515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61018c-d044-4539-a079-25676f3dbe73 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.034937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61018c-d044-4539-a079-25676f3dbe73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.061209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-be61018c-d044-4539-a079-25676f3dbe73 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:59.062748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-be61018c-d044-4539-a079-25676f3dbe73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:59.090221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 181/352] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:59.091369 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525529b-2ed3-4cd7-81dc-0cf810e3a932 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:59.097020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:59.109778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:59.171920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 172/353] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:59.174027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176acdc2-9404-4848-a161-bb434634657b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.174350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176acdc2-9404-4848-a161-bb434634657b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.174630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176acdc2-9404-4848-a161-bb434634657b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.180935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 173/354] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:59.180935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176acdc2-9404-4848-a161-bb434634657b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:59.245951 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525529b-2ed3-4cd7-81dc-0cf810e3a932 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:59.295382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a279ab0-c895-4390-98a4-99968ee6a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:59.296408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a279ab0-c895-4390-98a4-99968ee6a0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.297414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a279ab0-c895-4390-98a4-99968ee6a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.297911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a279ab0-c895-4390-98a4-99968ee6a0e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.312394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176acdc2-9404-4848-a161-bb434634657b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:59.357954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71da89-b134-4cf9-811a-bc16de40c02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:59.359667 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71da89-b134-4cf9-811a-bc16de40c02f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.360195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71da89-b134-4cf9-811a-bc16de40c02f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.360683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71da89-b134-4cf9-811a-bc16de40c02f None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.362491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71da89-b134-4cf9-811a-bc16de40c02f None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:59.364475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71da89-b134-4cf9-811a-bc16de40c02f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:59.367542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 174/355] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:59 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:59.369341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443cdff-e927-466f-a9ac-9f686b084035 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.369839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443cdff-e927-466f-a9ac-9f686b084035 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.374903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443cdff-e927-466f-a9ac-9f686b084035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.395821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c443cdff-e927-466f-a9ac-9f686b084035 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:51:59.396745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c443cdff-e927-466f-a9ac-9f686b084035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:51:59.398477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a279ab0-c895-4390-98a4-99968ee6a0e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:59.401365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a279ab0-c895-4390-98a4-99968ee6a0e7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:59.404140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:59.423734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:59.488945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 175/356] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:51:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:51:59.489625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443cdff-e927-466f-a9ac-9f686b084035 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:59.554776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 182/357] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:51:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:51:59.555565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a279ab0-c895-4390-98a4-99968ee6a0e7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:59.628600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443cdff-e927-466f-a9ac-9f686b084035 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:59.674146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25a7bc-164d-4bfc-b27a-34141562d4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:59.675255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25a7bc-164d-4bfc-b27a-34141562d4ab None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.675679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25a7bc-164d-4bfc-b27a-34141562d4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.676380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a25a7bc-164d-4bfc-b27a-34141562d4ab None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.690157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a25a7bc-164d-4bfc-b27a-34141562d4ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:59.697546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a279ab0-c895-4390-98a4-99968ee6a0e7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5a25a7bc-164d-4bfc-b27a-34141562d4ab None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:59.698623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:59.699677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:59.700668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 176/358] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:51:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:59.701371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25a7bc-164d-4bfc-b27a-34141562d4ab None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:59.702026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:59.731918 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:51:59.741709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb962fc3-2d38-46e4-a1d7-96fe183cf3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:59.742673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb962fc3-2d38-46e4-a1d7-96fe183cf3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.743029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb962fc3-2d38-46e4-a1d7-96fe183cf3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.743418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb962fc3-2d38-46e4-a1d7-96fe183cf3e1 None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.744631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb962fc3-2d38-46e4-a1d7-96fe183cf3e1 None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:59.746283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb962fc3-2d38-46e4-a1d7-96fe183cf3e1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:59.748014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 183/359] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:51:59 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:59.749097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb962fc3-2d38-46e4-a1d7-96fe183cf3e1 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:59.832854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a25a7bc-164d-4bfc-b27a-34141562d4ab None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:59.879247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb962fc3-2d38-46e4-a1d7-96fe183cf3e1 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:51:59.896446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46929a-576f-45c7-adef-51dc827129d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:59.897268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46929a-576f-45c7-adef-51dc827129d1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.897553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46929a-576f-45c7-adef-51dc827129d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.897836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46929a-576f-45c7-adef-51dc827129d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.916776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46929a-576f-45c7-adef-51dc827129d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:51:59.920897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5feb02-0a1b-4ef1-a1cc-0cd073ef7f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:51:59.921764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5feb02-0a1b-4ef1-a1cc-0cd073ef7f4d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.922120 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5feb02-0a1b-4ef1-a1cc-0cd073ef7f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.922369 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5feb02-0a1b-4ef1-a1cc-0cd073ef7f4d None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.923652 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5feb02-0a1b-4ef1-a1cc-0cd073ef7f4d None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:51:59.925094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5feb02-0a1b-4ef1-a1cc-0cd073ef7f4d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-cb46929a-576f-45c7-adef-51dc827129d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:51:59.925579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:51:59.926624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:51:59.927403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 177/360] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:51:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:51:59.928084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5d853-eca1-4606-8a2f-8cc3d55b27dd None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:51:59.928532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5d853-eca1-4606-8a2f-8cc3d55b27dd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:51:59.928624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f5d853-eca1-4606-8a2f-8cc3d55b27dd None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:51:59.930178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 178/361] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:51:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:51:59.931641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5d853-eca1-4606-8a2f-8cc3d55b27dd None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:59.932487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 184/362] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:51:59 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:51:59.933424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5feb02-0a1b-4ef1-a1cc-0cd073ef7f4d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:51:59.991382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:00.040062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:00.066880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5feb02-0a1b-4ef1-a1cc-0cd073ef7f4d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:00.070007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f5d853-eca1-4606-8a2f-8cc3d55b27dd None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:00.114083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d3850-a684-4536-8524-ccc43cbfb6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:00.114681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d3850-a684-4536-8524-ccc43cbfb6fd None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:00.114965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d3850-a684-4536-8524-ccc43cbfb6fd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:00.115542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d3850-a684-4536-8524-ccc43cbfb6fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:00.116395 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d3850-a684-4536-8524-ccc43cbfb6fd None admin] RBAC: Adding query filter params (group.id=af2ff3e5a07c4cf98efc807633fe9b8f, role.id=4f31bdaad0194bbd8fab39617059d14a, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:00.116711 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d3850-a684-4536-8524-ccc43cbfb6fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:00.118212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d3850-a684-4536-8524-ccc43cbfb6fd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:00.124691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ce1dd9-bd82-4522-b547-1974cc4a1381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:00.125389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce1dd9-bd82-4522-b547-1974cc4a1381 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:00.125578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce1dd9-bd82-4522-b547-1974cc4a1381 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:00.126466 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ce1dd9-bd82-4522-b547-1974cc4a1381 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:00.126567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 185/363] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Jun 26 02:51:59 2022] GET /identity/v3/role_assignments?group.id=af2ff3e5a07c4cf98efc807633fe9b8f&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=4f31bdaad0194bbd8fab39617059d14a => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:00.126657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce1dd9-bd82-4522-b547-1974cc4a1381 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:00.127086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce1dd9-bd82-4522-b547-1974cc4a1381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:00.128325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ce1dd9-bd82-4522-b547-1974cc4a1381 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:00.130855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d3850-a684-4536-8524-ccc43cbfb6fd None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:00.145710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 179/364] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:51:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:00.147252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b598b3-ba7f-4422-be2d-f42d67f411d9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:00.147690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b598b3-ba7f-4422-be2d-f42d67f411d9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:00.148093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b598b3-ba7f-4422-be2d-f42d67f411d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:00.168338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-71b598b3-ba7f-4422-be2d-f42d67f411d9 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:00.169715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-71b598b3-ba7f-4422-be2d-f42d67f411d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:00.255528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d3850-a684-4536-8524-ccc43cbfb6fd None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:00.257682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:00.261226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 180/365] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:00.267931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b598b3-ba7f-4422-be2d-f42d67f411d9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:00.321650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf90ed7-ac01-4269-a3e5-f5436c484837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:00.322926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf90ed7-ac01-4269-a3e5-f5436c484837 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:00.324809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf90ed7-ac01-4269-a3e5-f5436c484837 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:00.325925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf90ed7-ac01-4269-a3e5-f5436c484837 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:00.327564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf90ed7-ac01-4269-a3e5-f5436c484837 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:00.328615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf90ed7-ac01-4269-a3e5-f5436c484837 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:00.332613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf90ed7-ac01-4269-a3e5-f5436c484837 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:00.345349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 186/366] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:00.346273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf90ed7-ac01-4269-a3e5-f5436c484837 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:00.377480 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:00.421622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b598b3-ba7f-4422-be2d-f42d67f411d9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:00.468028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18529e-43d0-4854-981e-86fff719a28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:00.468631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18529e-43d0-4854-981e-86fff719a28e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:00.468912 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18529e-43d0-4854-981e-86fff719a28e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:00.469183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a18529e-43d0-4854-981e-86fff719a28e None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:00.470491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18529e-43d0-4854-981e-86fff719a28e None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:00.471922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a18529e-43d0-4854-981e-86fff719a28e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:00.473269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf90ed7-ac01-4269-a3e5-f5436c484837 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:00.473573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 181/367] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:00 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:00.474353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18529e-43d0-4854-981e-86fff719a28e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:00.523676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af7aa4-6ad9-4625-a490-32e34bd542ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:00.524913 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af7aa4-6ad9-4625-a490-32e34bd542ef None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:00.525408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af7aa4-6ad9-4625-a490-32e34bd542ef None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:00.525905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af7aa4-6ad9-4625-a490-32e34bd542ef None admin] PATH_INFO: `/v3/users/917b28b15e9c412a95d92b078b3c662f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:00.527732 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af7aa4-6ad9-4625-a490-32e34bd542ef None admin] RBAC: Authorizing `identity:get_user(user_id=917b28b15e9c412a95d92b078b3c662f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:00.529173 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af7aa4-6ad9-4625-a490-32e34bd542ef None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:00.531701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 187/368] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:52:00 2022] GET /identity/v3/users/917b28b15e9c412a95d92b078b3c662f => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:00.532301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af7aa4-6ad9-4625-a490-32e34bd542ef None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:00.576498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:00.605801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a18529e-43d0-4854-981e-86fff719a28e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:00.647421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e3f4c-98aa-4704-ba9c-c7160f56411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:00.648668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e3f4c-98aa-4704-ba9c-c7160f56411b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:00.649125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e3f4c-98aa-4704-ba9c-c7160f56411b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:00.649608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e3f4c-98aa-4704-ba9c-c7160f56411b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:00.658462 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:00.664548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e3f4c-98aa-4704-ba9c-c7160f56411b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:00.667276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af7aa4-6ad9-4625-a490-32e34bd542ef None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1b6e3f4c-98aa-4704-ba9c-c7160f56411b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:00.674515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:00.675422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:00.676187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:00.678759 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 182/369] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:00.679763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e3f4c-98aa-4704-ba9c-c7160f56411b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:00.717470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1273ea-05e0-4ea9-b439-48680d074e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:00.718271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1273ea-05e0-4ea9-b439-48680d074e80 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:00.718561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1273ea-05e0-4ea9-b439-48680d074e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:00.718838 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1273ea-05e0-4ea9-b439-48680d074e80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:00.806486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1273ea-05e0-4ea9-b439-48680d074e80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:00.807968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1273ea-05e0-4ea9-b439-48680d074e80 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:00.821729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e3f4c-98aa-4704-ba9c-c7160f56411b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:00.842739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:00.868552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e2441-679f-4cca-b510-c4aa7e9b10be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:00.869231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e2441-679f-4cca-b510-c4aa7e9b10be None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:00.869520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e2441-679f-4cca-b510-c4aa7e9b10be None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:00.869802 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e2441-679f-4cca-b510-c4aa7e9b10be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:00.871157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e2441-679f-4cca-b510-c4aa7e9b10be None admin] RBAC: Adding query filter params (group.id=af2ff3e5a07c4cf98efc807633fe9b8f, role.id=e1ed407e2d924fac8012250bb2ecf94f, scope.project.id=65124292b1924cc889516053c00bda74) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:00.871484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e2441-679f-4cca-b510-c4aa7e9b10be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:00.872942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366e2441-679f-4cca-b510-c4aa7e9b10be None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:00.886637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 183/370] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Jun 26 02:52:00 2022] GET /identity/v3/role_assignments?group.id=af2ff3e5a07c4cf98efc807633fe9b8f&scope.project.id=65124292b1924cc889516053c00bda74&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:00.886637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e2441-679f-4cca-b510-c4aa7e9b10be None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:00.991174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 188/371] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:00.992656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1273ea-05e0-4ea9-b439-48680d074e80 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:01.043190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:01.129743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1273ea-05e0-4ea9-b439-48680d074e80 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:01.131783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e2441-679f-4cca-b510-c4aa7e9b10be None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:01.166996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:01.175392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64383f68-3911-4a9c-9886-07d4d6b8a0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:01.176182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64383f68-3911-4a9c-9886-07d4d6b8a0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:01.176484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64383f68-3911-4a9c-9886-07d4d6b8a0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:01.176781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64383f68-3911-4a9c-9886-07d4d6b8a0c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:01.177808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64383f68-3911-4a9c-9886-07d4d6b8a0c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:01.178163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64383f68-3911-4a9c-9886-07d4d6b8a0c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:01.180169 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64383f68-3911-4a9c-9886-07d4d6b8a0c6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:01.180967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a6673-41a3-4127-bd92-fdd455fd06bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:01.181791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a6673-41a3-4127-bd92-fdd455fd06bf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:01.182256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a6673-41a3-4127-bd92-fdd455fd06bf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:01.182691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a6673-41a3-4127-bd92-fdd455fd06bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:01.184242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a6673-41a3-4127-bd92-fdd455fd06bf None admin] RBAC: Adding query filter params (role.id=c2c12c04e72a4116bf5d8e58c2cedc7b, user.id=917b28b15e9c412a95d92b078b3c662f, scope.system=all) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:01.184698 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a6673-41a3-4127-bd92-fdd455fd06bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:01.186289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a6673-41a3-4127-bd92-fdd455fd06bf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:01.192926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 189/372] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:01.193839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64383f68-3911-4a9c-9886-07d4d6b8a0c6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:01.203745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 184/373] 10.222.0.13 () {56 vars in 1308 bytes} [Sun Jun 26 02:52:00 2022] GET /identity/v3/role_assignments?user.id=917b28b15e9c412a95d92b078b3c662f&scope.system=all&role.id=c2c12c04e72a4116bf5d8e58c2cedc7b => generated 510 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:01.204825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a6673-41a3-4127-bd92-fdd455fd06bf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:01.330924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64383f68-3911-4a9c-9886-07d4d6b8a0c6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:01.390311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d0f38-193f-456e-8b7a-94d6a7d352a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:01.391588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d0f38-193f-456e-8b7a-94d6a7d352a2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:01.392056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d0f38-193f-456e-8b7a-94d6a7d352a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:01.392510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d0f38-193f-456e-8b7a-94d6a7d352a2 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:01.394647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d0f38-193f-456e-8b7a-94d6a7d352a2 None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:01.396319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d0f38-193f-456e-8b7a-94d6a7d352a2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:01.398674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 190/374] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:01 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:01.404329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d0f38-193f-456e-8b7a-94d6a7d352a2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:01.418094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a6673-41a3-4127-bd92-fdd455fd06bf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:01.465955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:01.493381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906594e2-2abe-4f58-bf8b-ed91d840c1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:01.494079 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906594e2-2abe-4f58-bf8b-ed91d840c1a9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:01.494370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906594e2-2abe-4f58-bf8b-ed91d840c1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:01.494650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906594e2-2abe-4f58-bf8b-ed91d840c1a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:01.533240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:01.590073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d0f38-193f-456e-8b7a-94d6a7d352a2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:01.613991 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906594e2-2abe-4f58-bf8b-ed91d840c1a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:01.616532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906594e2-2abe-4f58-bf8b-ed91d840c1a9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:01.648181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8d0330-1885-4153-82b9-5d9d19c531b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:01.649646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d0330-1885-4153-82b9-5d9d19c531b8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:01.650505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d0330-1885-4153-82b9-5d9d19c531b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:01.650965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8d0330-1885-4153-82b9-5d9d19c531b8 None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:01.655007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d0330-1885-4153-82b9-5d9d19c531b8 None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:01.657996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8d0330-1885-4153-82b9-5d9d19c531b8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:01.661661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 191/375] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:01 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:01.664486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f1400-4ce9-47a3-a0bf-ffab1a320b62 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:01.665454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f1400-4ce9-47a3-a0bf-ffab1a320b62 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:01.670475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f1400-4ce9-47a3-a0bf-ffab1a320b62 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:01.671628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 192/376] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:01 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:01.675928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f1400-4ce9-47a3-a0bf-ffab1a320b62 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:01.764377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 185/377] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:01.766074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c88b06-f4ad-4927-a8c5-7169f21326e3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:01.766593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c88b06-f4ad-4927-a8c5-7169f21326e3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:01.767065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c88b06-f4ad-4927-a8c5-7169f21326e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:01.792826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-21c88b06-f4ad-4927-a8c5-7169f21326e3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:01.794406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-21c88b06-f4ad-4927-a8c5-7169f21326e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:01.814405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:01.827872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f1400-4ce9-47a3-a0bf-ffab1a320b62 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:01.867980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:01.876448 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aac815-a1b4-46f1-bfd9-869ffe764fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:01.877245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aac815-a1b4-46f1-bfd9-869ffe764fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:01.877599 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aac815-a1b4-46f1-bfd9-869ffe764fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:01.877945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aac815-a1b4-46f1-bfd9-869ffe764fa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:01.878961 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aac815-a1b4-46f1-bfd9-869ffe764fa5 None admin] RBAC: Adding query filter params (group.id=af2ff3e5a07c4cf98efc807633fe9b8f, role.id=4f31bdaad0194bbd8fab39617059d14a, scope.project.id=65124292b1924cc889516053c00bda74) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:01.879371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aac815-a1b4-46f1-bfd9-869ffe764fa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:01.880922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aac815-a1b4-46f1-bfd9-869ffe764fa5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:01.887996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 193/378] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Jun 26 02:52:01 2022] GET /identity/v3/role_assignments?group.id=af2ff3e5a07c4cf98efc807633fe9b8f&scope.project.id=65124292b1924cc889516053c00bda74&role.id=4f31bdaad0194bbd8fab39617059d14a => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:01.892918 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aac815-a1b4-46f1-bfd9-869ffe764fa5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:01.896575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 186/379] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:01.933792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c88b06-f4ad-4927-a8c5-7169f21326e3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:02.078413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c88b06-f4ad-4927-a8c5-7169f21326e3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:02.080991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aac815-a1b4-46f1-bfd9-869ffe764fa5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:02.138571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff694d2-f01a-4a72-ad2f-8f64aafd01a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:02.139396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff694d2-f01a-4a72-ad2f-8f64aafd01a2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:02.139676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff694d2-f01a-4a72-ad2f-8f64aafd01a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:02.139954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff694d2-f01a-4a72-ad2f-8f64aafd01a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:02.160311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8b342-b359-4346-9c63-e2a27871a2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:02.160661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8b342-b359-4346-9c63-e2a27871a2d7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:02.160885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8b342-b359-4346-9c63-e2a27871a2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:02.161161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e8b342-b359-4346-9c63-e2a27871a2d7 None admin] PATH_INFO: `/v3/projects/becb24f8723f432e9a3c4c740a191a0b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:02.162760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8b342-b359-4346-9c63-e2a27871a2d7 None admin] RBAC: Authorizing `identity:get_project(project_id=becb24f8723f432e9a3c4c740a191a0b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:02.164198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e8b342-b359-4346-9c63-e2a27871a2d7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:02.166112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 194/380] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:01 2022] GET /identity/v3/projects/becb24f8723f432e9a3c4c740a191a0b => generated 344 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:02.167032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff694d2-f01a-4a72-ad2f-8f64aafd01a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:02.175232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8b342-b359-4346-9c63-e2a27871a2d7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1ff694d2-f01a-4a72-ad2f-8f64aafd01a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:02.175662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:02.176683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:52:02.177457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:02.178251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 187/381] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:02.184895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff694d2-f01a-4a72-ad2f-8f64aafd01a2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:02.202023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:02.267148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:02.316638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff694d2-f01a-4a72-ad2f-8f64aafd01a2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:02.352236 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8f6f9-9a1f-4d92-933c-06ae3328fafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:02.353408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8f6f9-9a1f-4d92-933c-06ae3328fafc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:02.353829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8f6f9-9a1f-4d92-933c-06ae3328fafc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:02.354411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c8f6f9-9a1f-4d92-933c-06ae3328fafc None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:02.355946 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8f6f9-9a1f-4d92-933c-06ae3328fafc None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:02.356391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8f6f9-9a1f-4d92-933c-06ae3328fafc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:02.357922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c8f6f9-9a1f-4d92-933c-06ae3328fafc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:02.367494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 188/382] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:52:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:02.368180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e8b342-b359-4346-9c63-e2a27871a2d7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:02.373317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8f6f9-9a1f-4d92-933c-06ae3328fafc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:02.466518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:02.482769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547757e-b499-4d49-9389-de818b45a9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:02.483691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547757e-b499-4d49-9389-de818b45a9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:02.483929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547757e-b499-4d49-9389-de818b45a9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:02.484209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547757e-b499-4d49-9389-de818b45a9e6 None admin] PATH_INFO: `/v3/groups/cbd13d97a03d4f98a0156cc50bb57341` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:02.490643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c8f6f9-9a1f-4d92-933c-06ae3328fafc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:02.507413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547757e-b499-4d49-9389-de818b45a9e6 None admin] RBAC: Authorizing `identity:get_group(group_id=cbd13d97a03d4f98a0156cc50bb57341)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:02.508860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547757e-b499-4d49-9389-de818b45a9e6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:02.511187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 195/383] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:02 2022] GET /identity/v3/groups/cbd13d97a03d4f98a0156cc50bb57341 => generated 229 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:02.518999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547757e-b499-4d49-9389-de818b45a9e6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:02.530609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e50b71-6f37-4863-9297-f5fa83f99980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:02.531722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50b71-6f37-4863-9297-f5fa83f99980 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:02.532206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50b71-6f37-4863-9297-f5fa83f99980 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:02.532656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e50b71-6f37-4863-9297-f5fa83f99980 None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:02.538703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e50b71-6f37-4863-9297-f5fa83f99980 None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:02.540424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e50b71-6f37-4863-9297-f5fa83f99980 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:02.544564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 189/384] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:02 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:02.548697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e50b71-6f37-4863-9297-f5fa83f99980 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:02.646836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547757e-b499-4d49-9389-de818b45a9e6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:02.675808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:02.686324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0b709b-c87f-4e0b-8d59-466bc0e9e678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:02.686986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b709b-c87f-4e0b-8d59-466bc0e9e678 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:02.687273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b709b-c87f-4e0b-8d59-466bc0e9e678 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:02.687548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0b709b-c87f-4e0b-8d59-466bc0e9e678 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:02.688384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b709b-c87f-4e0b-8d59-466bc0e9e678 None admin] RBAC: Adding query filter params (group.id=cbd13d97a03d4f98a0156cc50bb57341, role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=becb24f8723f432e9a3c4c740a191a0b) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:02.688743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b709b-c87f-4e0b-8d59-466bc0e9e678 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:02.692248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0b709b-c87f-4e0b-8d59-466bc0e9e678 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:02.700373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 196/385] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Jun 26 02:52:02 2022] GET /identity/v3/role_assignments?group.id=cbd13d97a03d4f98a0156cc50bb57341&scope.project.id=becb24f8723f432e9a3c4c740a191a0b&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 260 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:02.700802 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e50b71-6f37-4863-9297-f5fa83f99980 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:02.701733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd242b-e8b6-47f4-9593-5bee5215325f None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:02.702028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd242b-e8b6-47f4-9593-5bee5215325f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:02.702364 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcd242b-e8b6-47f4-9593-5bee5215325f None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:02.703654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 197/386] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:02.716842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c23a4-dd6e-4f69-8ee8-d70fa65ef44f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:02.717190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c23a4-dd6e-4f69-8ee8-d70fa65ef44f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:02.717566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763c23a4-dd6e-4f69-8ee8-d70fa65ef44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:02.754721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825a058-d747-422a-88d2-04de7eb3bba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:02.755907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a058-d747-422a-88d2-04de7eb3bba6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:02.756358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a058-d747-422a-88d2-04de7eb3bba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:02.756773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c825a058-d747-422a-88d2-04de7eb3bba6 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:02.757740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-763c23a4-dd6e-4f69-8ee8-d70fa65ef44f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:02.758556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825a058-d747-422a-88d2-04de7eb3bba6 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:02.760730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c825a058-d747-422a-88d2-04de7eb3bba6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:02.762397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-763c23a4-dd6e-4f69-8ee8-d70fa65ef44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:02.766641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 190/387] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:02 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:02.772783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825a058-d747-422a-88d2-04de7eb3bba6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:02.797409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:02.880953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 198/388] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:02.905719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c23a4-dd6e-4f69-8ee8-d70fa65ef44f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:02.941123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c825a058-d747-422a-88d2-04de7eb3bba6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:02.992504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0f53c-b4b0-47d5-b1d3-9469e59cf10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:02.993707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0f53c-b4b0-47d5-b1d3-9469e59cf10b None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:02.994202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0f53c-b4b0-47d5-b1d3-9469e59cf10b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:02.994619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a0f53c-b4b0-47d5-b1d3-9469e59cf10b None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/groups/af2ff3e5a07c4cf98efc807633fe9b8f/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:02.996970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0f53c-b4b0-47d5-b1d3-9469e59cf10b None admin] RBAC: Authorizing `identity:create_grant(project_id=e855e8bdc61c4a8a91cadb04fe89906c, group_id=af2ff3e5a07c4cf98efc807633fe9b8f, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:02.998837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a0f53c-b4b0-47d5-b1d3-9469e59cf10b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:03.019798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 191/389] 10.222.0.13 () {58 vars in 1442 bytes} [Sun Jun 26 02:52:02 2022] PUT /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/groups/af2ff3e5a07c4cf98efc807633fe9b8f/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 02:52:03.037010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763c23a4-dd6e-4f69-8ee8-d70fa65ef44f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:03.062459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:03.083508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88154d72-e1d9-464e-9e3e-e7881704a169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:03.084351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88154d72-e1d9-464e-9e3e-e7881704a169 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:03.084638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88154d72-e1d9-464e-9e3e-e7881704a169 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:03.084920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88154d72-e1d9-464e-9e3e-e7881704a169 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:03.099253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88154d72-e1d9-464e-9e3e-e7881704a169 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-88154d72-e1d9-464e-9e3e-e7881704a169 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:03.107767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:03.108900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:52:03.109860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:03.114093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 199/390] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:03.115503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0f53c-b4b0-47d5-b1d3-9469e59cf10b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:03.213795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:03.256350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a0f53c-b4b0-47d5-b1d3-9469e59cf10b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:03.296148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684956e5-3b9f-4d7f-9a8a-9d6eb2a55d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:03.297246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684956e5-3b9f-4d7f-9a8a-9d6eb2a55d9e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:03.297736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684956e5-3b9f-4d7f-9a8a-9d6eb2a55d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:03.298261 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684956e5-3b9f-4d7f-9a8a-9d6eb2a55d9e None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:03.299213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684956e5-3b9f-4d7f-9a8a-9d6eb2a55d9e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:03.299716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684956e5-3b9f-4d7f-9a8a-9d6eb2a55d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:03.301274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684956e5-3b9f-4d7f-9a8a-9d6eb2a55d9e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:03.311200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 192/391] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:03.318888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88154d72-e1d9-464e-9e3e-e7881704a169 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:03.453726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88154d72-e1d9-464e-9e3e-e7881704a169 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:03.496295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce745715-cf9b-4387-8196-c75dd4812927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:03.497170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce745715-cf9b-4387-8196-c75dd4812927 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:03.497458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce745715-cf9b-4387-8196-c75dd4812927 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:03.497736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce745715-cf9b-4387-8196-c75dd4812927 None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:03.502967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce745715-cf9b-4387-8196-c75dd4812927 None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:03.504478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce745715-cf9b-4387-8196-c75dd4812927 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:03.506335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 200/392] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:03 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:03.512746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684956e5-3b9f-4d7f-9a8a-9d6eb2a55d9e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:03.526264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c42f16-72c5-433c-9827-8363b89dc663 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:03.526566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c42f16-72c5-433c-9827-8363b89dc663 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:03.526889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c42f16-72c5-433c-9827-8363b89dc663 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:03.528447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 201/393] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:03.539465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e4ac3-01d2-4a1e-b1c6-7b0348c89c4c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:03.539773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e4ac3-01d2-4a1e-b1c6-7b0348c89c4c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:03.540039 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e4ac3-01d2-4a1e-b1c6-7b0348c89c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:03.583090 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-141e4ac3-01d2-4a1e-b1c6-7b0348c89c4c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:03.584310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-141e4ac3-01d2-4a1e-b1c6-7b0348c89c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:03.630403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684956e5-3b9f-4d7f-9a8a-9d6eb2a55d9e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:03.660397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:03.671253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c646d4-ce69-4edc-99f6-085a431483a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:03.672120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c646d4-ce69-4edc-99f6-085a431483a9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:03.672646 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c646d4-ce69-4edc-99f6-085a431483a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:03.672923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c646d4-ce69-4edc-99f6-085a431483a9 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:03.674186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c646d4-ce69-4edc-99f6-085a431483a9 None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:03.675622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c646d4-ce69-4edc-99f6-085a431483a9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:03.683921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 193/394] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:03 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:03.687925 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c646d4-ce69-4edc-99f6-085a431483a9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:03.689870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:03.740177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 202/395] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:52:03.840748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c646d4-ce69-4edc-99f6-085a431483a9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:03.853833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e4ac3-01d2-4a1e-b1c6-7b0348c89c4c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:03.883026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45019f-01ad-4a9e-86b3-9a7ab6247133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:03.883919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45019f-01ad-4a9e-86b3-9a7ab6247133 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:03.884199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45019f-01ad-4a9e-86b3-9a7ab6247133 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:03.884472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d45019f-01ad-4a9e-86b3-9a7ab6247133 None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/groups/af2ff3e5a07c4cf98efc807633fe9b8f/roles/4f31bdaad0194bbd8fab39617059d14a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:03.887188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45019f-01ad-4a9e-86b3-9a7ab6247133 None admin] RBAC: Authorizing `identity:create_grant(project_id=e855e8bdc61c4a8a91cadb04fe89906c, group_id=af2ff3e5a07c4cf98efc807633fe9b8f, role_id=4f31bdaad0194bbd8fab39617059d14a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:03.888625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d45019f-01ad-4a9e-86b3-9a7ab6247133 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:03.925391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 194/396] 10.222.0.13 () {58 vars in 1442 bytes} [Sun Jun 26 02:52:03 2022] PUT /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c/groups/af2ff3e5a07c4cf98efc807633fe9b8f/roles/4f31bdaad0194bbd8fab39617059d14a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:04.075074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e4ac3-01d2-4a1e-b1c6-7b0348c89c4c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:04.118707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855afbf2-04e0-45a9-9371-40ede70daf1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:04.119517 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855afbf2-04e0-45a9-9371-40ede70daf1f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:04.119800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855afbf2-04e0-45a9-9371-40ede70daf1f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:04.120073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855afbf2-04e0-45a9-9371-40ede70daf1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:04.133100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855afbf2-04e0-45a9-9371-40ede70daf1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-855afbf2-04e0-45a9-9371-40ede70daf1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:04.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:04.142433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:04.143242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 203/397] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:04.150743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45019f-01ad-4a9e-86b3-9a7ab6247133 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:04.224560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:04.273191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d45019f-01ad-4a9e-86b3-9a7ab6247133 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:04.308553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce48c0-9ace-4440-8b75-f8a3ae6dc2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:04.309225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce48c0-9ace-4440-8b75-f8a3ae6dc2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:04.309508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce48c0-9ace-4440-8b75-f8a3ae6dc2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:04.309783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce48c0-9ace-4440-8b75-f8a3ae6dc2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:04.310555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce48c0-9ace-4440-8b75-f8a3ae6dc2d6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:04.310861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce48c0-9ace-4440-8b75-f8a3ae6dc2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:04.312220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdce48c0-9ace-4440-8b75-f8a3ae6dc2d6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:04.320809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 195/398] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:52:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:04.326472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855afbf2-04e0-45a9-9371-40ede70daf1f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:04.395005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:04.473720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855afbf2-04e0-45a9-9371-40ede70daf1f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:04.519802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd387f-cf94-449e-b704-388e59bd6451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:04.520977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd387f-cf94-449e-b704-388e59bd6451 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:04.521449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd387f-cf94-449e-b704-388e59bd6451 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:04.521910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bd387f-cf94-449e-b704-388e59bd6451 None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:04.523280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd387f-cf94-449e-b704-388e59bd6451 None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:04.524893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bd387f-cf94-449e-b704-388e59bd6451 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:04.526784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 204/399] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:04 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:04.532477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce48c0-9ace-4440-8b75-f8a3ae6dc2d6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:04.653345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdce48c0-9ace-4440-8b75-f8a3ae6dc2d6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:04.678120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:04.697531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b787826-a648-4257-94ef-0cc66e9665e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:04.698451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b787826-a648-4257-94ef-0cc66e9665e4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:04.698772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b787826-a648-4257-94ef-0cc66e9665e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:04.699053 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b787826-a648-4257-94ef-0cc66e9665e4 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:04.700295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b787826-a648-4257-94ef-0cc66e9665e4 None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:04.701818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b787826-a648-4257-94ef-0cc66e9665e4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:04.703588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 196/400] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:04 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:04.709102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd387f-cf94-449e-b704-388e59bd6451 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:04.848357 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bd387f-cf94-449e-b704-388e59bd6451 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:04.860666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:04.899655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb09019-99ad-41ad-8ce2-776269acffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:04.900541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09019-99ad-41ad-8ce2-776269acffe4 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:04.900828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09019-99ad-41ad-8ce2-776269acffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:04.901101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb09019-99ad-41ad-8ce2-776269acffe4 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74/groups/af2ff3e5a07c4cf98efc807633fe9b8f/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:04.903636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb09019-99ad-41ad-8ce2-776269acffe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=65124292b1924cc889516053c00bda74, group_id=af2ff3e5a07c4cf98efc807633fe9b8f, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:04.905022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb09019-99ad-41ad-8ce2-776269acffe4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:04.931533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 205/401] 10.222.0.13 () {58 vars in 1442 bytes} [Sun Jun 26 02:52:04 2022] PUT /identity/v3/projects/65124292b1924cc889516053c00bda74/groups/af2ff3e5a07c4cf98efc807633fe9b8f/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:05.020739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff6743-f938-4f3e-bb12-aef34114aeac None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.021024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff6743-f938-4f3e-bb12-aef34114aeac None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.021485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff6743-f938-4f3e-bb12-aef34114aeac None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.023338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 197/402] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:05.029597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb29b2-7551-4f8a-9fb7-17aac046780b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.029970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb29b2-7551-4f8a-9fb7-17aac046780b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.030294 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eb29b2-7551-4f8a-9fb7-17aac046780b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.049036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-81eb29b2-7551-4f8a-9fb7-17aac046780b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:05.050271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-81eb29b2-7551-4f8a-9fb7-17aac046780b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:05.124256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 206/403] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:05.147149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff6743-f938-4f3e-bb12-aef34114aeac None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:05.260574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ff6743-f938-4f3e-bb12-aef34114aeac None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:05.296469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab6fdb5-7381-43fa-ab41-0c6298c7a3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:05.297298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6fdb5-7381-43fa-ab41-0c6298c7a3a5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.297586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6fdb5-7381-43fa-ab41-0c6298c7a3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.297864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab6fdb5-7381-43fa-ab41-0c6298c7a3a5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.308792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab6fdb5-7381-43fa-ab41-0c6298c7a3a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1ab6fdb5-7381-43fa-ab41-0c6298c7a3a5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:05.317069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:05.317956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:05.318763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 198/404] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:05.324886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb29b2-7551-4f8a-9fb7-17aac046780b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:05.331658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923af621-a7e8-4efb-b9ea-e01ac112fb53 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.331658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923af621-a7e8-4efb-b9ea-e01ac112fb53 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.331658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923af621-a7e8-4efb-b9ea-e01ac112fb53 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.331658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 199/405] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:05.340514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f769c8-93f9-43dc-865e-1bd2b93ca400 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.340514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f769c8-93f9-43dc-865e-1bd2b93ca400 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.340514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f769c8-93f9-43dc-865e-1bd2b93ca400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.367080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-87f769c8-93f9-43dc-865e-1bd2b93ca400 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:05.368625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-87f769c8-93f9-43dc-865e-1bd2b93ca400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:05.438322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:05.445625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eb29b2-7551-4f8a-9fb7-17aac046780b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:05.471540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:05.487218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44abe7c5-1efa-4e3c-82f0-d1306392b2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:05.487890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abe7c5-1efa-4e3c-82f0-d1306392b2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.488179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abe7c5-1efa-4e3c-82f0-d1306392b2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.488457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abe7c5-1efa-4e3c-82f0-d1306392b2a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.489381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44abe7c5-1efa-4e3c-82f0-d1306392b2a9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:05.489700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44abe7c5-1efa-4e3c-82f0-d1306392b2a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:05.491456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44abe7c5-1efa-4e3c-82f0-d1306392b2a9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:05.501776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 207/406] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:05.508302 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44abe7c5-1efa-4e3c-82f0-d1306392b2a9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:05.535282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 200/407] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:52:05.566023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f769c8-93f9-43dc-865e-1bd2b93ca400 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:05.655563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44abe7c5-1efa-4e3c-82f0-d1306392b2a9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:05.689768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6ab97-d063-45c7-badd-009313dfb56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:05.690664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6ab97-d063-45c7-badd-009313dfb56e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.690949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6ab97-d063-45c7-badd-009313dfb56e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.691221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6ab97-d063-45c7-badd-009313dfb56e None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.692426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6ab97-d063-45c7-badd-009313dfb56e None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:05.693852 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6ab97-d063-45c7-badd-009313dfb56e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:05.695867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 208/408] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:05 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:05.708114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6ab97-d063-45c7-badd-009313dfb56e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:05.729478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f769c8-93f9-43dc-865e-1bd2b93ca400 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:05.738405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:05.772572 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725f34c5-ea72-4bc0-9111-06d33c080050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:05.773394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f34c5-ea72-4bc0-9111-06d33c080050 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.773656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f34c5-ea72-4bc0-9111-06d33c080050 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.775067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725f34c5-ea72-4bc0-9111-06d33c080050 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.787653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725f34c5-ea72-4bc0-9111-06d33c080050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-725f34c5-ea72-4bc0-9111-06d33c080050 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:05.795704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:05.796600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:05.797388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 201/409] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:05.798564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31594d16-4cce-4e3f-805f-ad256a66f4bf None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.798847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31594d16-4cce-4e3f-805f-ad256a66f4bf None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.799123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31594d16-4cce-4e3f-805f-ad256a66f4bf None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.800639 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 202/410] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:05.802732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31594d16-4cce-4e3f-805f-ad256a66f4bf None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:05.837300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6ab97-d063-45c7-badd-009313dfb56e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:05.846458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:05.896635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e236126c-62d1-4f6a-bf2e-e1d36d9fcf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:05.898401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236126c-62d1-4f6a-bf2e-e1d36d9fcf85 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.898912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236126c-62d1-4f6a-bf2e-e1d36d9fcf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.899839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e236126c-62d1-4f6a-bf2e-e1d36d9fcf85 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.901707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236126c-62d1-4f6a-bf2e-e1d36d9fcf85 None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:05.904340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e236126c-62d1-4f6a-bf2e-e1d36d9fcf85 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:05.907565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 209/411] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:05 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:05.908934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8626d8-f258-4896-8d09-d430f4f490cf None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.909403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8626d8-f258-4896-8d09-d430f4f490cf None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.909852 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8626d8-f258-4896-8d09-d430f4f490cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.929381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31594d16-4cce-4e3f-805f-ad256a66f4bf None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:05.931245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3f8626d8-f258-4896-8d09-d430f4f490cf None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:05.933541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3f8626d8-f258-4896-8d09-d430f4f490cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:05.968377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072036f4-3232-48c0-be83-2e05ddd781c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:05.969183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072036f4-3232-48c0-be83-2e05ddd781c9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:05.969541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072036f4-3232-48c0-be83-2e05ddd781c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:05.969857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072036f4-3232-48c0-be83-2e05ddd781c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:05.970736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072036f4-3232-48c0-be83-2e05ddd781c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:05.971171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072036f4-3232-48c0-be83-2e05ddd781c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:05.972609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072036f4-3232-48c0-be83-2e05ddd781c9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:05.983081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 203/412] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:05.984814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072036f4-3232-48c0-be83-2e05ddd781c9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:06.030889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 210/413] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:06.032361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8626d8-f258-4896-8d09-d430f4f490cf None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:06.064243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:06.098549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:06.101965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072036f4-3232-48c0-be83-2e05ddd781c9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:06.140877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb6299-5a76-4a5c-8796-0f47f37a0175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:06.141928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb6299-5a76-4a5c-8796-0f47f37a0175 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:06.142499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb6299-5a76-4a5c-8796-0f47f37a0175 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:06.142819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eb6299-5a76-4a5c-8796-0f47f37a0175 None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74/groups/af2ff3e5a07c4cf98efc807633fe9b8f/roles/4f31bdaad0194bbd8fab39617059d14a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:06.145205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb6299-5a76-4a5c-8796-0f47f37a0175 None admin] RBAC: Authorizing `identity:create_grant(project_id=65124292b1924cc889516053c00bda74, group_id=af2ff3e5a07c4cf98efc807633fe9b8f, role_id=4f31bdaad0194bbd8fab39617059d14a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:06.146976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eb6299-5a76-4a5c-8796-0f47f37a0175 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:06.162636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 204/414] 10.222.0.13 () {58 vars in 1442 bytes} [Sun Jun 26 02:52:05 2022] PUT /identity/v3/projects/65124292b1924cc889516053c00bda74/groups/af2ff3e5a07c4cf98efc807633fe9b8f/roles/4f31bdaad0194bbd8fab39617059d14a => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:06.163938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb6299-5a76-4a5c-8796-0f47f37a0175 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:06.167026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8626d8-f258-4896-8d09-d430f4f490cf None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:06.226085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5899cc0-b391-468a-87e0-67c729d37a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:06.227510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5899cc0-b391-468a-87e0-67c729d37a1f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:06.227964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5899cc0-b391-468a-87e0-67c729d37a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:06.228497 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5899cc0-b391-468a-87e0-67c729d37a1f None admin] PATH_INFO: `/v3/groups/cbd13d97a03d4f98a0156cc50bb57341` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:06.230143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5899cc0-b391-468a-87e0-67c729d37a1f None admin] RBAC: Authorizing `identity:get_group(group_id=cbd13d97a03d4f98a0156cc50bb57341)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:06.231852 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5899cc0-b391-468a-87e0-67c729d37a1f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:06.234094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 211/415] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:06 2022] GET /identity/v3/groups/cbd13d97a03d4f98a0156cc50bb57341 => generated 229 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:06.239439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5899cc0-b391-468a-87e0-67c729d37a1f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:06.302200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eb6299-5a76-4a5c-8796-0f47f37a0175 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:06.342607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b166f-798b-460f-ad61-0581d49f3c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:06.343505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b166f-798b-460f-ad61-0581d49f3c16 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:06.343898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b166f-798b-460f-ad61-0581d49f3c16 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:06.344273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b166f-798b-460f-ad61-0581d49f3c16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:06.356921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814b166f-798b-460f-ad61-0581d49f3c16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-814b166f-798b-460f-ad61-0581d49f3c16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:06.366932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:06.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:06.368695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 205/416] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:06.376549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b166f-798b-460f-ad61-0581d49f3c16 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:06.403694 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:06.420231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:06.437012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5899cc0-b391-468a-87e0-67c729d37a1f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:06.483257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155836b2-93ff-45b6-9e83-ee2167ebaeb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:06.484510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155836b2-93ff-45b6-9e83-ee2167ebaeb4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:06.485021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155836b2-93ff-45b6-9e83-ee2167ebaeb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:06.485509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155836b2-93ff-45b6-9e83-ee2167ebaeb4 None admin] PATH_INFO: `/v3/projects/becb24f8723f432e9a3c4c740a191a0b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:06.488798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155836b2-93ff-45b6-9e83-ee2167ebaeb4 None admin] RBAC: Authorizing `identity:get_project(project_id=becb24f8723f432e9a3c4c740a191a0b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:06.490621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155836b2-93ff-45b6-9e83-ee2167ebaeb4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:06.492914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 212/417] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:06 2022] GET /identity/v3/projects/becb24f8723f432e9a3c4c740a191a0b => generated 344 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:06.498204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad312ff-28f7-42fc-bb27-54448abf97ad None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:06.498204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad312ff-28f7-42fc-bb27-54448abf97ad None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:06.498204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad312ff-28f7-42fc-bb27-54448abf97ad None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:06.498204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 213/418] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:06.502765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58219-2832-494f-a2b7-7246593a0aa0 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:06.504381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58219-2832-494f-a2b7-7246593a0aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:06.504673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd58219-2832-494f-a2b7-7246593a0aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:06.514968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b166f-798b-460f-ad61-0581d49f3c16 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:06.527164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1cd58219-2832-494f-a2b7-7246593a0aa0 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:06.528896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1cd58219-2832-494f-a2b7-7246593a0aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:06.555704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2323442-6c45-497d-85fe-862f4ae22192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:06.556437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2323442-6c45-497d-85fe-862f4ae22192 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:06.556730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2323442-6c45-497d-85fe-862f4ae22192 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:06.557013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2323442-6c45-497d-85fe-862f4ae22192 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:06.558676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2323442-6c45-497d-85fe-862f4ae22192 None admin] RBAC: Adding query filter params (name=member) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:06.558998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2323442-6c45-497d-85fe-862f4ae22192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:06.560630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2323442-6c45-497d-85fe-862f4ae22192 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:06.574128 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 206/419] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:52:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:06.575219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2323442-6c45-497d-85fe-862f4ae22192 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:06.618210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 214/420] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:52:06.619270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd58219-2832-494f-a2b7-7246593a0aa0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:06.696404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:06.716621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2323442-6c45-497d-85fe-862f4ae22192 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:06.752841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b30065-2a9d-418a-8234-ee29f9ee1992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:06.753915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b30065-2a9d-418a-8234-ee29f9ee1992 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:06.754498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b30065-2a9d-418a-8234-ee29f9ee1992 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:06.754667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:06.755304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b30065-2a9d-418a-8234-ee29f9ee1992 None admin] PATH_INFO: `/v3/projects/becb24f8723f432e9a3c4c740a191a0b/groups/cbd13d97a03d4f98a0156cc50bb57341/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:06.757625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b30065-2a9d-418a-8234-ee29f9ee1992 None admin] RBAC: Authorizing `identity:create_grant(project_id=becb24f8723f432e9a3c4c740a191a0b, group_id=cbd13d97a03d4f98a0156cc50bb57341, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:06.759347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b30065-2a9d-418a-8234-ee29f9ee1992 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:06.768149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd58219-2832-494f-a2b7-7246593a0aa0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:06.779447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 207/421] 10.222.0.13 () {58 vars in 1442 bytes} [Sun Jun 26 02:52:06 2022] PUT /identity/v3/projects/becb24f8723f432e9a3c4c740a191a0b/groups/cbd13d97a03d4f98a0156cc50bb57341/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:06.781431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b30065-2a9d-418a-8234-ee29f9ee1992 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:06.807222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bf48a-0b43-4e4f-8768-7f6f8689ef68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:06.808530 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf48a-0b43-4e4f-8768-7f6f8689ef68 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:06.808818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf48a-0b43-4e4f-8768-7f6f8689ef68 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:06.809152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18bf48a-0b43-4e4f-8768-7f6f8689ef68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:06.897336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bf48a-0b43-4e4f-8768-7f6f8689ef68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:06.898910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18bf48a-0b43-4e4f-8768-7f6f8689ef68 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:06.930356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b30065-2a9d-418a-8234-ee29f9ee1992 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:06.994105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05a54c-149e-4e07-9a7a-f47d0c2ad243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:06.994958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05a54c-149e-4e07-9a7a-f47d0c2ad243 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:06.995246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05a54c-149e-4e07-9a7a-f47d0c2ad243 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:06.995572 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa05a54c-149e-4e07-9a7a-f47d0c2ad243 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:07.023296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa05a54c-149e-4e07-9a7a-f47d0c2ad243 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:07.028208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-aa05a54c-149e-4e07-9a7a-f47d0c2ad243 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:07.035796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:07.036707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:07.038131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 208/422] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:07.039912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 215/423] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:07.045239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05a54c-149e-4e07-9a7a-f47d0c2ad243 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:07.045484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bf48a-0b43-4e4f-8768-7f6f8689ef68 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:07.048911 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:07.191858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18bf48a-0b43-4e4f-8768-7f6f8689ef68 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:07.203652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa05a54c-149e-4e07-9a7a-f47d0c2ad243 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:07.256745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189dc70-903b-47fb-988a-ad0b4b2466cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:07.257612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189dc70-903b-47fb-988a-ad0b4b2466cb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:07.257894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189dc70-903b-47fb-988a-ad0b4b2466cb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:07.258197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189dc70-903b-47fb-988a-ad0b4b2466cb None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:07.259880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf793e9-b0d0-4e03-9894-96108f629240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:07.260544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf793e9-b0d0-4e03-9894-96108f629240 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:07.260819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf793e9-b0d0-4e03-9894-96108f629240 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:07.261123 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf793e9-b0d0-4e03-9894-96108f629240 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:07.261874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf793e9-b0d0-4e03-9894-96108f629240 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:07.262204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf793e9-b0d0-4e03-9894-96108f629240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:07.263018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189dc70-903b-47fb-988a-ad0b4b2466cb None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:07.265691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf793e9-b0d0-4e03-9894-96108f629240 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:07.266288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189dc70-903b-47fb-988a-ad0b4b2466cb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:07.270078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 209/424] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:07 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:07.273887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189dc70-903b-47fb-988a-ad0b4b2466cb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:07.281524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 216/425] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:07.287729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf793e9-b0d0-4e03-9894-96108f629240 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:07.362587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:07.408794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:07.414855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf793e9-b0d0-4e03-9894-96108f629240 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:07.449471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bed849-2e21-4ed2-bd2d-92b23589555c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:07.450065 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bed849-2e21-4ed2-bd2d-92b23589555c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:07.450345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bed849-2e21-4ed2-bd2d-92b23589555c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:07.450619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bed849-2e21-4ed2-bd2d-92b23589555c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:07.480405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189dc70-903b-47fb-988a-ad0b4b2466cb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:07.526104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bed849-2e21-4ed2-bd2d-92b23589555c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:07.528929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bed849-2e21-4ed2-bd2d-92b23589555c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:07.538989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64d363-088b-4b65-956f-60d12418c453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:07.539920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64d363-088b-4b65-956f-60d12418c453 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:07.540203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64d363-088b-4b65-956f-60d12418c453 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:07.540479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64d363-088b-4b65-956f-60d12418c453 None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:07.542433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64d363-088b-4b65-956f-60d12418c453 None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:07.544334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64d363-088b-4b65-956f-60d12418c453 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:07.546673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 210/426] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:07 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:07.548716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0c94a-681c-4266-9b09-1fadd39ded0a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:07.549009 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0c94a-681c-4266-9b09-1fadd39ded0a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:07.549290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f0c94a-681c-4266-9b09-1fadd39ded0a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:07.551105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 211/427] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:07.552082 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0c94a-681c-4266-9b09-1fadd39ded0a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:07.592622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:07.663881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 217/428] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:07.665577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88686e-ebf1-4c03-9e33-aeaadcfe5591 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:07.666332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88686e-ebf1-4c03-9e33-aeaadcfe5591 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:07.666850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd88686e-ebf1-4c03-9e33-aeaadcfe5591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:07.675153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f0c94a-681c-4266-9b09-1fadd39ded0a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:07.685027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dd88686e-ebf1-4c03-9e33-aeaadcfe5591 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:07.686535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dd88686e-ebf1-4c03-9e33-aeaadcfe5591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:07.719934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75762b5f-d0c5-48a8-8360-63ce490788f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:07.720641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75762b5f-d0c5-48a8-8360-63ce490788f3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:07.720927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75762b5f-d0c5-48a8-8360-63ce490788f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:07.721203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75762b5f-d0c5-48a8-8360-63ce490788f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:07.722265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75762b5f-d0c5-48a8-8360-63ce490788f3 None admin] RBAC: Adding query filter params (group.id=af2ff3e5a07c4cf98efc807633fe9b8f, role.id=e1ed407e2d924fac8012250bb2ecf94f, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:07.722592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75762b5f-d0c5-48a8-8360-63ce490788f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:07.724032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75762b5f-d0c5-48a8-8360-63ce490788f3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:07.736561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 212/429] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Jun 26 02:52:07 2022] GET /identity/v3/role_assignments?group.id=af2ff3e5a07c4cf98efc807633fe9b8f&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:07.737461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75762b5f-d0c5-48a8-8360-63ce490788f3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:07.741229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:07.765836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 218/430] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:07.801080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88686e-ebf1-4c03-9e33-aeaadcfe5591 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:07.877752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75762b5f-d0c5-48a8-8360-63ce490788f3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:07.931702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f641c9-3942-4f7e-b8f9-048d4c288a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:07.932569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f641c9-3942-4f7e-b8f9-048d4c288a7a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:07.933379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f641c9-3942-4f7e-b8f9-048d4c288a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:07.933654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f641c9-3942-4f7e-b8f9-048d4c288a7a None admin] PATH_INFO: `/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:07.935313 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f641c9-3942-4f7e-b8f9-048d4c288a7a None admin] RBAC: Authorizing `identity:get_project(project_id=e855e8bdc61c4a8a91cadb04fe89906c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:07.936740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f641c9-3942-4f7e-b8f9-048d4c288a7a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:07.939457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 213/431] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:07 2022] GET /identity/v3/projects/e855e8bdc61c4a8a91cadb04fe89906c => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:07.945051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f641c9-3942-4f7e-b8f9-048d4c288a7a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:07.952842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:07.976233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd88686e-ebf1-4c03-9e33-aeaadcfe5591 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:08.014033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabfe09-05ed-4296-8621-5754f69fb708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:08.015222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabfe09-05ed-4296-8621-5754f69fb708 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:08.015734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabfe09-05ed-4296-8621-5754f69fb708 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:08.016224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dabfe09-05ed-4296-8621-5754f69fb708 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:08.023229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:08.027809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dabfe09-05ed-4296-8621-5754f69fb708 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1dabfe09-05ed-4296-8621-5754f69fb708 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:08.035812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:08.036710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 26 02:52:08.037492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:08.040135 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 219/432] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:08.045986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabfe09-05ed-4296-8621-5754f69fb708 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:08.064135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f641c9-3942-4f7e-b8f9-048d4c288a7a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:08.112975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b744d2d-61c5-4c86-a8e1-dc67b9037024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:08.113892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b744d2d-61c5-4c86-a8e1-dc67b9037024 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:08.114217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b744d2d-61c5-4c86-a8e1-dc67b9037024 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:08.114493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b744d2d-61c5-4c86-a8e1-dc67b9037024 None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:08.115685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b744d2d-61c5-4c86-a8e1-dc67b9037024 None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:08.117177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b744d2d-61c5-4c86-a8e1-dc67b9037024 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:08.119181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 214/433] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:07 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:08.125415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b744d2d-61c5-4c86-a8e1-dc67b9037024 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:08.190748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dabfe09-05ed-4296-8621-5754f69fb708 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:08.231965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213f9ae-f204-46f9-85ec-b17af0d3f847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:08.232586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213f9ae-f204-46f9-85ec-b17af0d3f847 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:08.232868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213f9ae-f204-46f9-85ec-b17af0d3f847 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:08.233141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c213f9ae-f204-46f9-85ec-b17af0d3f847 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:08.233944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213f9ae-f204-46f9-85ec-b17af0d3f847 None admin] RBAC: Adding query filter params (name=member) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:08.234313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213f9ae-f204-46f9-85ec-b17af0d3f847 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:08.235803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c213f9ae-f204-46f9-85ec-b17af0d3f847 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:08.250304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 220/434] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:52:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:08.252177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b744d2d-61c5-4c86-a8e1-dc67b9037024 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:08.255734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213f9ae-f204-46f9-85ec-b17af0d3f847 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:08.260567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:08.296059 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b01816-3bb7-42f3-934c-4081c77f9e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:08.296741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b01816-3bb7-42f3-934c-4081c77f9e4f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:08.297025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b01816-3bb7-42f3-934c-4081c77f9e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:08.297300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b01816-3bb7-42f3-934c-4081c77f9e4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:08.298162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b01816-3bb7-42f3-934c-4081c77f9e4f None admin] RBAC: Adding query filter params (group.id=af2ff3e5a07c4cf98efc807633fe9b8f, role.id=4f31bdaad0194bbd8fab39617059d14a, scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:08.298476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b01816-3bb7-42f3-934c-4081c77f9e4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:08.300214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b01816-3bb7-42f3-934c-4081c77f9e4f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:08.314026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 215/435] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Jun 26 02:52:08 2022] GET /identity/v3/role_assignments?group.id=af2ff3e5a07c4cf98efc807633fe9b8f&scope.project.id=e855e8bdc61c4a8a91cadb04fe89906c&role.id=4f31bdaad0194bbd8fab39617059d14a => generated 611 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:08.378591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c213f9ae-f204-46f9-85ec-b17af0d3f847 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:08.417860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea61643-35b6-40a2-ac06-9fbfc11c7d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:08.418558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea61643-35b6-40a2-ac06-9fbfc11c7d3a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:08.418844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea61643-35b6-40a2-ac06-9fbfc11c7d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:08.419117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea61643-35b6-40a2-ac06-9fbfc11c7d3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:08.494664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea61643-35b6-40a2-ac06-9fbfc11c7d3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:08.496091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea61643-35b6-40a2-ac06-9fbfc11c7d3a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:08.499155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:08.570072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3593182-6a10-4e85-b385-52504ed5ecf6 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:08.570464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3593182-6a10-4e85-b385-52504ed5ecf6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:08.570750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3593182-6a10-4e85-b385-52504ed5ecf6 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:08.572157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 216/436] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:08.577494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6b4c0-3a57-404b-92cd-7d6ab3c8e4ab None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:08.577792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6b4c0-3a57-404b-92cd-7d6ab3c8e4ab None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:08.578174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6b4c0-3a57-404b-92cd-7d6ab3c8e4ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:08.591898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:08.605279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 221/437] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:08.610490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2fe6b4c0-3a57-404b-92cd-7d6ab3c8e4ab None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:08.610839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea61643-35b6-40a2-ac06-9fbfc11c7d3a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:08.613454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2fe6b4c0-3a57-404b-92cd-7d6ab3c8e4ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:08.709364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 217/438] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:08.733737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6b4c0-3a57-404b-92cd-7d6ab3c8e4ab None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:08.736106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea61643-35b6-40a2-ac06-9fbfc11c7d3a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:08.740040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:08.783423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112a680-0246-4b6c-a320-29233fda890e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:08.784332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112a680-0246-4b6c-a320-29233fda890e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:08.784620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112a680-0246-4b6c-a320-29233fda890e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:08.784900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112a680-0246-4b6c-a320-29233fda890e None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:08.786136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112a680-0246-4b6c-a320-29233fda890e None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:08.787575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112a680-0246-4b6c-a320-29233fda890e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:08.789619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 222/439] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:08 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:08.795366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112a680-0246-4b6c-a320-29233fda890e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:08.865261 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6b4c0-3a57-404b-92cd-7d6ab3c8e4ab None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:08.897386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:08.903239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545d86-f966-495c-9df6-ce8258ef075f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:08.904159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545d86-f966-495c-9df6-ce8258ef075f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:08.904446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545d86-f966-495c-9df6-ce8258ef075f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:08.904726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28545d86-f966-495c-9df6-ce8258ef075f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:08.910619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112a680-0246-4b6c-a320-29233fda890e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:08.915234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28545d86-f966-495c-9df6-ce8258ef075f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-28545d86-f966-495c-9df6-ce8258ef075f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:08.923806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:08.924656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:52:08.925439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:08.929964 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 218/440] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:08.930818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545d86-f966-495c-9df6-ce8258ef075f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:08.945814 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f4899-0fca-48c2-9a89-92d8e100ecd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:08.946701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f4899-0fca-48c2-9a89-92d8e100ecd4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:08.946985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f4899-0fca-48c2-9a89-92d8e100ecd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:08.947262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494f4899-0fca-48c2-9a89-92d8e100ecd4 None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:08.948471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f4899-0fca-48c2-9a89-92d8e100ecd4 None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:08.949903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494f4899-0fca-48c2-9a89-92d8e100ecd4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:08.956154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 223/441] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:08 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:08.957182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f4899-0fca-48c2-9a89-92d8e100ecd4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:09.001330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:09.040435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28545d86-f966-495c-9df6-ce8258ef075f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:09.065649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494f4899-0fca-48c2-9a89-92d8e100ecd4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:09.077191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c7b60-99e3-4e8c-a5e7-7ce2c6332378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:09.077857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c7b60-99e3-4e8c-a5e7-7ce2c6332378 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.078175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c7b60-99e3-4e8c-a5e7-7ce2c6332378 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.078455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c7b60-99e3-4e8c-a5e7-7ce2c6332378 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.079186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c7b60-99e3-4e8c-a5e7-7ce2c6332378 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:09.079488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c7b60-99e3-4e8c-a5e7-7ce2c6332378 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:09.080854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c7b60-99e3-4e8c-a5e7-7ce2c6332378 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:09.089751 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 219/442] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:09.095045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c7b60-99e3-4e8c-a5e7-7ce2c6332378 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:09.105152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e608bbcb-a884-42ab-9a97-70a9da0499e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:09.105829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608bbcb-a884-42ab-9a97-70a9da0499e3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.106193 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608bbcb-a884-42ab-9a97-70a9da0499e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.106476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e608bbcb-a884-42ab-9a97-70a9da0499e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.107312 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608bbcb-a884-42ab-9a97-70a9da0499e3 None admin] RBAC: Adding query filter params (group.id=af2ff3e5a07c4cf98efc807633fe9b8f, role.id=e1ed407e2d924fac8012250bb2ecf94f, scope.project.id=65124292b1924cc889516053c00bda74) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:09.107629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608bbcb-a884-42ab-9a97-70a9da0499e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:09.109068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e608bbcb-a884-42ab-9a97-70a9da0499e3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:09.115418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 224/443] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Jun 26 02:52:08 2022] GET /identity/v3/role_assignments?group.id=af2ff3e5a07c4cf98efc807633fe9b8f&scope.project.id=65124292b1924cc889516053c00bda74&role.id=e1ed407e2d924fac8012250bb2ecf94f => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:09.160172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:09.210117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c7b60-99e3-4e8c-a5e7-7ce2c6332378 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:09.245290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e108-f9af-4d1c-944c-b6de5b9f7918 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.245590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e108-f9af-4d1c-944c-b6de5b9f7918 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.245872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e108-f9af-4d1c-944c-b6de5b9f7918 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.247323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 225/444] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:09.247509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12843200-eb90-45fc-9cc8-97c3e6755d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:09.248324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12843200-eb90-45fc-9cc8-97c3e6755d61 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.248998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12843200-eb90-45fc-9cc8-97c3e6755d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.249413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12843200-eb90-45fc-9cc8-97c3e6755d61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.253085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158d26d-e7f0-48fd-a8e5-820c1dff882c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.253374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158d26d-e7f0-48fd-a8e5-820c1dff882c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.253650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e158d26d-e7f0-48fd-a8e5-820c1dff882c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.271753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e158d26d-e7f0-48fd-a8e5-820c1dff882c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:09.273187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e158d26d-e7f0-48fd-a8e5-820c1dff882c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:09.327754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12843200-eb90-45fc-9cc8-97c3e6755d61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:09.329145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12843200-eb90-45fc-9cc8-97c3e6755d61 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:09.358159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 226/445] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:52:09.381358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158d26d-e7f0-48fd-a8e5-820c1dff882c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:09.388281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:09.428335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:09.439029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 220/446] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 26 02:52:09.444752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12843200-eb90-45fc-9cc8-97c3e6755d61 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:09.502116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e158d26d-e7f0-48fd-a8e5-820c1dff882c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:09.535781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76a8c2-4762-4bb2-a607-217ae96bc2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:09.536568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76a8c2-4762-4bb2-a607-217ae96bc2db None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.536857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76a8c2-4762-4bb2-a607-217ae96bc2db None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.537132 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76a8c2-4762-4bb2-a607-217ae96bc2db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.547818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed76a8c2-4762-4bb2-a607-217ae96bc2db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:09.549299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12843200-eb90-45fc-9cc8-97c3e6755d61 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ed76a8c2-4762-4bb2-a607-217ae96bc2db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:09.555495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:09.556447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:09.557216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 227/447] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:09.563159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76a8c2-4762-4bb2-a607-217ae96bc2db None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:09.585572 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71342438-f55f-4426-b480-a88021a38aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:09.586485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71342438-f55f-4426-b480-a88021a38aaf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.586963 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71342438-f55f-4426-b480-a88021a38aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.587375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71342438-f55f-4426-b480-a88021a38aaf None admin] PATH_INFO: `/v3/projects/65124292b1924cc889516053c00bda74` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.588812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71342438-f55f-4426-b480-a88021a38aaf None admin] RBAC: Authorizing `identity:get_project(project_id=65124292b1924cc889516053c00bda74)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:09.590543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71342438-f55f-4426-b480-a88021a38aaf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:09.592311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 221/448] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:09 2022] GET /identity/v3/projects/65124292b1924cc889516053c00bda74 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:09.597732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71342438-f55f-4426-b480-a88021a38aaf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:09.622313 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:09.669112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:09.676163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76a8c2-4762-4bb2-a607-217ae96bc2db None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:09.696175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71342438-f55f-4426-b480-a88021a38aaf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:09.709669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c044c0f-56ac-4370-95d2-ff5f5b48245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:09.710355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c044c0f-56ac-4370-95d2-ff5f5b48245f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.710639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c044c0f-56ac-4370-95d2-ff5f5b48245f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.710912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c044c0f-56ac-4370-95d2-ff5f5b48245f None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.711661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c044c0f-56ac-4370-95d2-ff5f5b48245f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:09.711959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c044c0f-56ac-4370-95d2-ff5f5b48245f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:09.713321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c044c0f-56ac-4370-95d2-ff5f5b48245f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:09.721113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 228/449] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:09.726745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c044c0f-56ac-4370-95d2-ff5f5b48245f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:09.728719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17952ccd-a803-4e08-8d29-f1de482196b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:09.729558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17952ccd-a803-4e08-8d29-f1de482196b0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.730395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17952ccd-a803-4e08-8d29-f1de482196b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.730675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17952ccd-a803-4e08-8d29-f1de482196b0 None admin] PATH_INFO: `/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.731862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17952ccd-a803-4e08-8d29-f1de482196b0 None admin] RBAC: Authorizing `identity:get_group(group_id=af2ff3e5a07c4cf98efc807633fe9b8f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:09.733277 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17952ccd-a803-4e08-8d29-f1de482196b0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:09.739031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 222/450] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:09 2022] GET /identity/v3/groups/af2ff3e5a07c4cf98efc807633fe9b8f => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:09.740491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17952ccd-a803-4e08-8d29-f1de482196b0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:09.829450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:09.831302 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c044c0f-56ac-4370-95d2-ff5f5b48245f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:09.839298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17952ccd-a803-4e08-8d29-f1de482196b0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:09.865391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd7101-62e4-4912-9227-6ada492da997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:09.866123 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd7101-62e4-4912-9227-6ada492da997 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.866414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd7101-62e4-4912-9227-6ada492da997 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.866694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd7101-62e4-4912-9227-6ada492da997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.878780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f0056-3e5e-43f6-aadd-fdeede1b1eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:09.879427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f0056-3e5e-43f6-aadd-fdeede1b1eaf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:09.879954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f0056-3e5e-43f6-aadd-fdeede1b1eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:09.879954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832f0056-3e5e-43f6-aadd-fdeede1b1eaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:09.880806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f0056-3e5e-43f6-aadd-fdeede1b1eaf None admin] RBAC: Adding query filter params (group.id=af2ff3e5a07c4cf98efc807633fe9b8f, role.id=4f31bdaad0194bbd8fab39617059d14a, scope.project.id=65124292b1924cc889516053c00bda74) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:09.881121 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f0056-3e5e-43f6-aadd-fdeede1b1eaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:09.882601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832f0056-3e5e-43f6-aadd-fdeede1b1eaf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:09.888392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 223/451] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Jun 26 02:52:09 2022] GET /identity/v3/role_assignments?group.id=af2ff3e5a07c4cf98efc807633fe9b8f&scope.project.id=65124292b1924cc889516053c00bda74&role.id=4f31bdaad0194bbd8fab39617059d14a => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:09.892386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:09.939430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd7101-62e4-4912-9227-6ada492da997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:09.940939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bd7101-62e4-4912-9227-6ada492da997 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:10.048355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 229/452] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 26 02:52:10.054025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f0056-3e5e-43f6-aadd-fdeede1b1eaf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:10.157471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832f0056-3e5e-43f6-aadd-fdeede1b1eaf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:10.191061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19600c10-2aae-46a6-b355-5dbe4233cefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:10.191935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19600c10-2aae-46a6-b355-5dbe4233cefb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:10.192214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19600c10-2aae-46a6-b355-5dbe4233cefb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:10.192490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19600c10-2aae-46a6-b355-5dbe4233cefb None admin] PATH_INFO: `/v3/projects/becb24f8723f432e9a3c4c740a191a0b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:10.193687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19600c10-2aae-46a6-b355-5dbe4233cefb None admin] RBAC: Authorizing `identity:get_project(project_id=becb24f8723f432e9a3c4c740a191a0b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:10.195155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19600c10-2aae-46a6-b355-5dbe4233cefb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:10.196883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 224/453] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:52:10 2022] GET /identity/v3/projects/becb24f8723f432e9a3c4c740a191a0b => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:10.202448 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd7101-62e4-4912-9227-6ada492da997 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:10.260005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:10.303301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd7101-62e4-4912-9227-6ada492da997 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:10.335526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df1961a-5785-4a21-8859-9a40f3139f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:10.336369 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1961a-5785-4a21-8859-9a40f3139f57 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:10.336656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1961a-5785-4a21-8859-9a40f3139f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:10.336932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1961a-5785-4a21-8859-9a40f3139f57 None admin] PATH_INFO: `/v3/groups/cbd13d97a03d4f98a0156cc50bb57341` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:10.338186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1961a-5785-4a21-8859-9a40f3139f57 None admin] RBAC: Authorizing `identity:get_group(group_id=cbd13d97a03d4f98a0156cc50bb57341)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:10.340200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df1961a-5785-4a21-8859-9a40f3139f57 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:10.341849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 230/454] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Jun 26 02:52:10 2022] GET /identity/v3/groups/cbd13d97a03d4f98a0156cc50bb57341 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:10.347560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19600c10-2aae-46a6-b355-5dbe4233cefb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:10.371500 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:10.444932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19600c10-2aae-46a6-b355-5dbe4233cefb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:10.477773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a9ef58-6950-498b-aac5-938c96f989e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:10.478437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9ef58-6950-498b-aac5-938c96f989e8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:10.478723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9ef58-6950-498b-aac5-938c96f989e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:10.478999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a9ef58-6950-498b-aac5-938c96f989e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:10.479857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9ef58-6950-498b-aac5-938c96f989e8 None admin] RBAC: Adding query filter params (group.id=cbd13d97a03d4f98a0156cc50bb57341, role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=becb24f8723f432e9a3c4c740a191a0b) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:10.480171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9ef58-6950-498b-aac5-938c96f989e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:10.481580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a9ef58-6950-498b-aac5-938c96f989e8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:10.487345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 225/455] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Jun 26 02:52:10 2022] GET /identity/v3/role_assignments?group.id=cbd13d97a03d4f98a0156cc50bb57341&scope.project.id=becb24f8723f432e9a3c4c740a191a0b&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:12.913857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ebc57-f71e-4743-a53f-f37c911212e8 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:12.914384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ebc57-f71e-4743-a53f-f37c911212e8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:12.914456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2ebc57-f71e-4743-a53f-f37c911212e8 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:12.915977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 231/456] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:12.921549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d2f46-3e33-4c4b-b703-6f941cf9c88e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:12.922417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d2f46-3e33-4c4b-b703-6f941cf9c88e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:12.922945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d2f46-3e33-4c4b-b703-6f941cf9c88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:12.946980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ed2d2f46-3e33-4c4b-b703-6f941cf9c88e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:12.948529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ed2d2f46-3e33-4c4b-b703-6f941cf9c88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:13.044399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 226/457] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:13.089345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ebc57-f71e-4743-a53f-f37c911212e8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:13.235694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2ebc57-f71e-4743-a53f-f37c911212e8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:13.263564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbdb6a-c078-4bed-8a8d-8516fc36ef06 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:13.264179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbdb6a-c078-4bed-8a8d-8516fc36ef06 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:13.264666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbdb6a-c078-4bed-8a8d-8516fc36ef06 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:13.266301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 227/458] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:13.270309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:13.271569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8b49-b7e0-4a3d-a22c-64f061e11969 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:13.272063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8b49-b7e0-4a3d-a22c-64f061e11969 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:13.272597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8b49-b7e0-4a3d-a22c-64f061e11969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:13.279156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34579259-c194-40c2-a7e9-1b29bf5ba10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:13.279156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34579259-c194-40c2-a7e9-1b29bf5ba10e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:13.279156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34579259-c194-40c2-a7e9-1b29bf5ba10e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:13.279156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34579259-c194-40c2-a7e9-1b29bf5ba10e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:13.296863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34579259-c194-40c2-a7e9-1b29bf5ba10e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:13.298559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34579259-c194-40c2-a7e9-1b29bf5ba10e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:13.311017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a5ab8b49-b7e0-4a3d-a22c-64f061e11969 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:13.312384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a5ab8b49-b7e0-4a3d-a22c-64f061e11969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-34579259-c194-40c2-a7e9-1b29bf5ba10e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:13.331322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:13.332391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:13.333447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:13.333447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:13.333447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:13.333447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:13.339819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 232/459] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:13.346127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:13.346258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34579259-c194-40c2-a7e9-1b29bf5ba10e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:13.441729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 228/460] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:52:13.453270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ade6e-ea51-4cc5-b065-e10e0adfab4d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:13.453566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ade6e-ea51-4cc5-b065-e10e0adfab4d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:13.453942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ade6e-ea51-4cc5-b065-e10e0adfab4d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:13.455616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 229/461] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:13.463017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6403c-c1ee-4d84-bd0b-fe3ad9bf5676 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:13.463305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6403c-c1ee-4d84-bd0b-fe3ad9bf5676 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:13.463587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6403c-c1ee-4d84-bd0b-fe3ad9bf5676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:13.484343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-67d6403c-c1ee-4d84-bd0b-fe3ad9bf5676 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:13.485548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-67d6403c-c1ee-4d84-bd0b-fe3ad9bf5676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:13.515983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34579259-c194-40c2-a7e9-1b29bf5ba10e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:13.563444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddc434-5ba2-48d0-8ff3-33d84e2dd309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:13.570063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 230/462] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:52:13.570063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6403c-c1ee-4d84-bd0b-fe3ad9bf5676 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:13.570726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddc434-5ba2-48d0-8ff3-33d84e2dd309 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:13.571018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddc434-5ba2-48d0-8ff3-33d84e2dd309 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:13.571299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ddc434-5ba2-48d0-8ff3-33d84e2dd309 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:13.572053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddc434-5ba2-48d0-8ff3-33d84e2dd309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:13.572366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddc434-5ba2-48d0-8ff3-33d84e2dd309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:13.579263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ddc434-5ba2-48d0-8ff3-33d84e2dd309 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:13.601288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 233/463] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:13.602527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddc434-5ba2-48d0-8ff3-33d84e2dd309 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:13.646733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:13.666300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:13.690753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6403c-c1ee-4d84-bd0b-fe3ad9bf5676 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:13.724728 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ddc434-5ba2-48d0-8ff3-33d84e2dd309 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:13.726459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed798c-fb51-45f1-a72e-85dda2e97c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:13.727274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed798c-fb51-45f1-a72e-85dda2e97c53 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:13.727564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed798c-fb51-45f1-a72e-85dda2e97c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:13.727844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faed798c-fb51-45f1-a72e-85dda2e97c53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:13.739535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed798c-fb51-45f1-a72e-85dda2e97c53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:13.741008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faed798c-fb51-45f1-a72e-85dda2e97c53 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-faed798c-fb51-45f1-a72e-85dda2e97c53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:13.761916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:13.762821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:13.763742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:13.763742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:13.763742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:13.763742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:13.767163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2ecee-959e-4bcd-bf77-45fd09b41ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:13.768189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 231/464] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:13.768410 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2ecee-959e-4bcd-bf77-45fd09b41ad3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:13.768885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2ecee-959e-4bcd-bf77-45fd09b41ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:13.769106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed798c-fb51-45f1-a72e-85dda2e97c53 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:13.769546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2ecee-959e-4bcd-bf77-45fd09b41ad3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:13.781965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2ecee-959e-4bcd-bf77-45fd09b41ad3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:13.783777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2ecee-959e-4bcd-bf77-45fd09b41ad3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ebb2ecee-959e-4bcd-bf77-45fd09b41ad3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:13.802926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:13.803820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:13.804649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:13.804649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:13.804649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:13.804649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:13.811953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 234/465] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:13.813080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2ecee-959e-4bcd-bf77-45fd09b41ad3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:13.885817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faed798c-fb51-45f1-a72e-85dda2e97c53 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:13.910627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:13.914250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:13.922810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8455f-f5f4-4b05-861f-d9285b9b4af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:13.924187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8455f-f5f4-4b05-861f-d9285b9b4af0 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:13.924670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8455f-f5f4-4b05-861f-d9285b9b4af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:13.925068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2ecee-959e-4bcd-bf77-45fd09b41ad3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:13.925475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d8455f-f5f4-4b05-861f-d9285b9b4af0 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:13.925849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8455f-f5f4-4b05-861f-d9285b9b4af0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:13.927539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d8455f-f5f4-4b05-861f-d9285b9b4af0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:13.951022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 232/466] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:52:13 2022] POST /identity/v3/users => generated 246 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:13.952324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8455f-f5f4-4b05-861f-d9285b9b4af0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:13.974515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5985f0e-e207-4c5d-8e01-6748322c98b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:13.975650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5985f0e-e207-4c5d-8e01-6748322c98b2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:13.976209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5985f0e-e207-4c5d-8e01-6748322c98b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:13.976691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5985f0e-e207-4c5d-8e01-6748322c98b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:13.978503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5985f0e-e207-4c5d-8e01-6748322c98b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:13.979488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5985f0e-e207-4c5d-8e01-6748322c98b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:13.982151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5985f0e-e207-4c5d-8e01-6748322c98b2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:13.999883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 235/467] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:14.001548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f271bf-8be8-4387-94f0-2b500d159034 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.002000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f271bf-8be8-4387-94f0-2b500d159034 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.002454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f271bf-8be8-4387-94f0-2b500d159034 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.008959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 236/468] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:14.008959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f271bf-8be8-4387-94f0-2b500d159034 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:14.080517 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d8455f-f5f4-4b05-861f-d9285b9b4af0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:14.115959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f91559-476a-43a0-ada9-c45e8448b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:14.116628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91559-476a-43a0-ada9-c45e8448b800 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.116912 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91559-476a-43a0-ada9-c45e8448b800 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.117187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f91559-476a-43a0-ada9-c45e8448b800 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.117917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f91559-476a-43a0-ada9-c45e8448b800 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:14.118249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f91559-476a-43a0-ada9-c45e8448b800 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:14.119641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f91559-476a-43a0-ada9-c45e8448b800 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:14.132761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 233/469] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:14.134093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f271bf-8be8-4387-94f0-2b500d159034 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:14.134547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed68d71-0456-4e16-9d6a-04c8e7fec47d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.134898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed68d71-0456-4e16-9d6a-04c8e7fec47d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.135181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed68d71-0456-4e16-9d6a-04c8e7fec47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.152956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1ed68d71-0456-4e16-9d6a-04c8e7fec47d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:14.154259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1ed68d71-0456-4e16-9d6a-04c8e7fec47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:14.170166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3aaebc-2e2e-4da0-8fa9-353ed1451911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:14.170833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3aaebc-2e2e-4da0-8fa9-353ed1451911 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.171116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3aaebc-2e2e-4da0-8fa9-353ed1451911 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.171393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3aaebc-2e2e-4da0-8fa9-353ed1451911 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.172096 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3aaebc-2e2e-4da0-8fa9-353ed1451911 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:14.173524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3aaebc-2e2e-4da0-8fa9-353ed1451911 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:14.194326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 237/470] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:52:14 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:14.195618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b6ac6-555e-4041-aacf-715c305316fa None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.195907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b6ac6-555e-4041-aacf-715c305316fa None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.196181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947b6ac6-555e-4041-aacf-715c305316fa None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.200807 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:14.202419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 238/471] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:14.203366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b6ac6-555e-4041-aacf-715c305316fa None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:14.207469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:14.238906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 234/472] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:14.240560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222e397-0af1-47b4-a3a4-ccd76d49764b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.240994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222e397-0af1-47b4-a3a4-ccd76d49764b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.241400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8222e397-0af1-47b4-a3a4-ccd76d49764b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.277665 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8222e397-0af1-47b4-a3a4-ccd76d49764b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:14.279086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8222e397-0af1-47b4-a3a4-ccd76d49764b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:14.343409 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947b6ac6-555e-4041-aacf-715c305316fa None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:14.361821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 235/473] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:14.362977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222e397-0af1-47b4-a3a4-ccd76d49764b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:14.383623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d62f0-aa09-467b-bf54-71a7b3d1d797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:14.384680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d62f0-aa09-467b-bf54-71a7b3d1d797 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.384972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d62f0-aa09-467b-bf54-71a7b3d1d797 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.385250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d62f0-aa09-467b-bf54-71a7b3d1d797 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.386552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d62f0-aa09-467b-bf54-71a7b3d1d797 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:14.387946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d62f0-aa09-467b-bf54-71a7b3d1d797 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:14.410664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 239/474] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:52:14 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:14.411857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d62f0-aa09-467b-bf54-71a7b3d1d797 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:14.488057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:14.501522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8222e397-0af1-47b4-a3a4-ccd76d49764b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:14.505676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:14.543972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d62f0-aa09-467b-bf54-71a7b3d1d797 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:14.550168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c58aca-3ea5-4763-8a75-7636faeee15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:14.551345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c58aca-3ea5-4763-8a75-7636faeee15f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.552491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c58aca-3ea5-4763-8a75-7636faeee15f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.552968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c58aca-3ea5-4763-8a75-7636faeee15f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.569738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c58aca-3ea5-4763-8a75-7636faeee15f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:14.572142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c58aca-3ea5-4763-8a75-7636faeee15f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:14.590452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55132d-60f8-405a-b420-f65ab6400c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:14.591174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55132d-60f8-405a-b420-f65ab6400c03 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.591465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55132d-60f8-405a-b420-f65ab6400c03 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.591774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55132d-60f8-405a-b420-f65ab6400c03 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.592636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55132d-60f8-405a-b420-f65ab6400c03 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:14.594184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55132d-60f8-405a-b420-f65ab6400c03 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-31c58aca-3ea5-4763-8a75-7636faeee15f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:14.598985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:14.600006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:14.600829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:14.600829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:14.600829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:14.600829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:14.601040 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 236/475] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:14.603474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.sql.core [None req-ea55132d-60f8-405a-b420-f65ab6400c03 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 26 02:52:14.603474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 26 02:52:14.603474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [parameters: {'id': '4b18d25012864ec0817f6c17f4adc97c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 26 02:52:14.603474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80103) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 26 02:52:14.606709 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c58aca-3ea5-4763-8a75-7636faeee15f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ea55132d-60f8-405a-b420-f65ab6400c03 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = self._query(query) Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application conn.query(q) Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result.read() Jun 26 02:52:14.622059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return ref.to_dict() Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 26 02:52:14.622986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application next(self.gen) Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application yield resource Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application next(self.gen) Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.session.rollback() Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.force_reraise() Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise self.value Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application session.commit() Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 26 02:52:14.624220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._prepare_impl() Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.session.flush() Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._flush(objects) Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application compat.raise_( Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application flush_context.execute() Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rec.execute(self) Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 26 02:52:14.625081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 26 02:52:14.625914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = self._query(query) Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application conn.query(q) Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result.read() Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 26 02:52:14.628399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [parameters: {'id': '4b18d25012864ec0817f6c17f4adc97c', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:14.630636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 26 02:52:14.634778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:14.637836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 240/476] 10.222.0.13 () {60 vars in 1131 bytes} [Sun Jun 26 02:52:14 2022] POST /identity/v3/roles => generated 144 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:14.637836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55132d-60f8-405a-b420-f65ab6400c03 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:14.746662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c58aca-3ea5-4763-8a75-7636faeee15f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:14.770331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55132d-60f8-405a-b420-f65ab6400c03 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:14.805454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ba326-3beb-426b-ad2e-3a572e8083db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:14.808322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ba326-3beb-426b-ad2e-3a572e8083db None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.808528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ba326-3beb-426b-ad2e-3a572e8083db None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.808800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070ba326-3beb-426b-ad2e-3a572e8083db None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.809526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ba326-3beb-426b-ad2e-3a572e8083db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:14.809829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ba326-3beb-426b-ad2e-3a572e8083db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:14.811242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070ba326-3beb-426b-ad2e-3a572e8083db None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:14.820578 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:14.833215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5c9a2-d393-4c00-adfa-939cc88a13bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:14.834126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5c9a2-d393-4c00-adfa-939cc88a13bc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:14.834425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5c9a2-d393-4c00-adfa-939cc88a13bc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:14.834702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f5c9a2-d393-4c00-adfa-939cc88a13bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:14.840797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 237/477] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:14.845887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ba326-3beb-426b-ad2e-3a572e8083db None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:14.862650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f5c9a2-d393-4c00-adfa-939cc88a13bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-20f5c9a2-d393-4c00-adfa-939cc88a13bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:14.875855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:14.876727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 26 02:52:14.877496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:14.880033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 241/478] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:14.887111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5c9a2-d393-4c00-adfa-939cc88a13bc None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:14.891909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:15.003240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f5c9a2-d393-4c00-adfa-939cc88a13bc None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:15.078828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1382402-3cbc-453a-be8d-f8b139d9e0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:15.079500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1382402-3cbc-453a-be8d-f8b139d9e0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:15.079781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1382402-3cbc-453a-be8d-f8b139d9e0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:15.080054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1382402-3cbc-453a-be8d-f8b139d9e0f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:15.080809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1382402-3cbc-453a-be8d-f8b139d9e0f1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:15.081119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1382402-3cbc-453a-be8d-f8b139d9e0f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:15.082538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1382402-3cbc-453a-be8d-f8b139d9e0f1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:15.092796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 242/479] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:15.097230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070ba326-3beb-426b-ad2e-3a572e8083db None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:15.135179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7911a2c-0933-4dd6-9f58-583220aac6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:15.136497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911a2c-0933-4dd6-9f58-583220aac6ed None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:15.136785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911a2c-0933-4dd6-9f58-583220aac6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:15.137063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7911a2c-0933-4dd6-9f58-583220aac6ed None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:15.137800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911a2c-0933-4dd6-9f58-583220aac6ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:15.139302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7911a2c-0933-4dd6-9f58-583220aac6ed None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:15.165704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 238/480] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:52:14 2022] POST /identity/v3/users => generated 249 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:52:16.442992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e6b39-d9d7-4a9c-925a-641ba70e1447 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:16.443973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e6b39-d9d7-4a9c-925a-641ba70e1447 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:16.444446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e6b39-d9d7-4a9c-925a-641ba70e1447 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:16.446313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 243/481] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:16.451563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184a94c-9c40-4231-8b5a-6ef2b5740330 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:16.452328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184a94c-9c40-4231-8b5a-6ef2b5740330 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:16.452815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0184a94c-9c40-4231-8b5a-6ef2b5740330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:16.474445 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0184a94c-9c40-4231-8b5a-6ef2b5740330 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:16.475933 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0184a94c-9c40-4231-8b5a-6ef2b5740330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:16.564668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 239/482] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:16.608379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e6b39-d9d7-4a9c-925a-641ba70e1447 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:16.654237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9589ef-b627-45d7-b25d-52a3f1afc1e5 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:16.654566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9589ef-b627-45d7-b25d-52a3f1afc1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:16.654813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9589ef-b627-45d7-b25d-52a3f1afc1e5 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:16.656475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 240/483] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:16.661935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538241df-b1c8-443b-bc44-684c87600cde None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:16.662274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538241df-b1c8-443b-bc44-684c87600cde None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:16.662532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538241df-b1c8-443b-bc44-684c87600cde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:16.682326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:16.687061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-538241df-b1c8-443b-bc44-684c87600cde None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:16.688288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-538241df-b1c8-443b-bc44-684c87600cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:16.702519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:16.767666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 241/484] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:52:16.791957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538241df-b1c8-443b-bc44-684c87600cde None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:16.796695 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e6b39-d9d7-4a9c-925a-641ba70e1447 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:16.870034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32ed7b-ec84-4798-85bd-d00f355df766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:16.870903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32ed7b-ec84-4798-85bd-d00f355df766 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:16.871189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32ed7b-ec84-4798-85bd-d00f355df766 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:16.871468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32ed7b-ec84-4798-85bd-d00f355df766 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:16.886876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32ed7b-ec84-4798-85bd-d00f355df766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:16.897552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538241df-b1c8-443b-bc44-684c87600cde None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ae32ed7b-ec84-4798-85bd-d00f355df766 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:16.902468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:16.903319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:16.904112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 244/485] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:16.909389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32ed7b-ec84-4798-85bd-d00f355df766 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:16.942393 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:16.953273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4133b-3424-4dc8-b707-295d8b2b9625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:16.954013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4133b-3424-4dc8-b707-295d8b2b9625 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:16.954321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4133b-3424-4dc8-b707-295d8b2b9625 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:16.954595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b4133b-3424-4dc8-b707-295d8b2b9625 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:16.978201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b4133b-3424-4dc8-b707-295d8b2b9625 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a8b4133b-3424-4dc8-b707-295d8b2b9625 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:16.985903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:16.987961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:16.989834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 242/486] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:16.997252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4133b-3424-4dc8-b707-295d8b2b9625 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:17.098672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32ed7b-ec84-4798-85bd-d00f355df766 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:17.114355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:17.161038 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff564e65-f28b-4990-9eeb-a323f60d9aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:17.161713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564e65-f28b-4990-9eeb-a323f60d9aef None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.162003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564e65-f28b-4990-9eeb-a323f60d9aef None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.162428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff564e65-f28b-4990-9eeb-a323f60d9aef None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.163299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff564e65-f28b-4990-9eeb-a323f60d9aef None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:17.163606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff564e65-f28b-4990-9eeb-a323f60d9aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:17.165003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff564e65-f28b-4990-9eeb-a323f60d9aef None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:17.179663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b4133b-3424-4dc8-b707-295d8b2b9625 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:17.188228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 245/487] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:17.189644 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95725a86-b9fa-45a2-a801-ea20fb695b5a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.189939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95725a86-b9fa-45a2-a801-ea20fb695b5a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.190248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95725a86-b9fa-45a2-a801-ea20fb695b5a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.192028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 246/488] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:17.192894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95725a86-b9fa-45a2-a801-ea20fb695b5a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:17.223857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e33c7ad-05f6-4887-9a86-a0d99fab32ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:17.225336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c7ad-05f6-4887-9a86-a0d99fab32ac None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.225920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c7ad-05f6-4887-9a86-a0d99fab32ac None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.226481 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e33c7ad-05f6-4887-9a86-a0d99fab32ac None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.227374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33c7ad-05f6-4887-9a86-a0d99fab32ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:17.227817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33c7ad-05f6-4887-9a86-a0d99fab32ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:17.230139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e33c7ad-05f6-4887-9a86-a0d99fab32ac None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:17.241931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 243/489] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:17.248530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a49d5-0dd9-4c91-af51-565ab9c95979 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.249081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a49d5-0dd9-4c91-af51-565ab9c95979 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.249355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a49d5-0dd9-4c91-af51-565ab9c95979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.266398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d67a49d5-0dd9-4c91-af51-565ab9c95979 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:17.267596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d67a49d5-0dd9-4c91-af51-565ab9c95979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:17.324825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95725a86-b9fa-45a2-a801-ea20fb695b5a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:17.344087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:17.363040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2a649-5cdc-41cf-addc-250bec50b028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:17.363934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a649-5cdc-41cf-addc-250bec50b028 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.364221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a649-5cdc-41cf-addc-250bec50b028 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.364494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac2a649-5cdc-41cf-addc-250bec50b028 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.369936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 244/490] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:17.371778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a49d5-0dd9-4c91-af51-565ab9c95979 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:17.377385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a649-5cdc-41cf-addc-250bec50b028 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:17.378896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac2a649-5cdc-41cf-addc-250bec50b028 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-4ac2a649-5cdc-41cf-addc-250bec50b028 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:17.397251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:17.398314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:17.399278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:17.399278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:17.399278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:17.399278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:17.403491 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 247/491] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:17.404950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2a649-5cdc-41cf-addc-250bec50b028 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:17.429888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:17.511968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a49d5-0dd9-4c91-af51-565ab9c95979 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:17.521384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac2a649-5cdc-41cf-addc-250bec50b028 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:17.559485 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9670a9-1d62-4259-8d16-ed6e45fafbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:17.560141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fa2e1-ad81-4609-8a9d-2fdf62f64f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:17.560427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9670a9-1d62-4259-8d16-ed6e45fafbc8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.560817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9670a9-1d62-4259-8d16-ed6e45fafbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.561196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9670a9-1d62-4259-8d16-ed6e45fafbc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.561578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa2e1-ad81-4609-8a9d-2fdf62f64f83 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.562016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa2e1-ad81-4609-8a9d-2fdf62f64f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.562290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9670a9-1d62-4259-8d16-ed6e45fafbc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:17.562599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0fa2e1-ad81-4609-8a9d-2fdf62f64f83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.562797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9670a9-1d62-4259-8d16-ed6e45fafbc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:17.564295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9670a9-1d62-4259-8d16-ed6e45fafbc8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:17.575867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fa2e1-ad81-4609-8a9d-2fdf62f64f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:17.577574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0fa2e1-ad81-4609-8a9d-2fdf62f64f83 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:17.582166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 248/492] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:17.583140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9670a9-1d62-4259-8d16-ed6e45fafbc8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-af0fa2e1-ad81-4609-8a9d-2fdf62f64f83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:17.601382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:17.602503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:17.603550 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:17.603550 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:17.603550 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:17.603550 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:17.607523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 245/493] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:17.613482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fa2e1-ad81-4609-8a9d-2fdf62f64f83 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:17.662246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:17.686699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:17.707218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9670a9-1d62-4259-8d16-ed6e45fafbc8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:17.738453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0fa2e1-ad81-4609-8a9d-2fdf62f64f83 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:17.741384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823deacf-779c-4c78-8150-13f2694f9f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:17.742209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823deacf-779c-4c78-8150-13f2694f9f42 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.742488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823deacf-779c-4c78-8150-13f2694f9f42 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.742760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823deacf-779c-4c78-8150-13f2694f9f42 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.753125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823deacf-779c-4c78-8150-13f2694f9f42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-823deacf-779c-4c78-8150-13f2694f9f42 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:17.760735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:17.761604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:17.762481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 249/494] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:17.763201 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823deacf-779c-4c78-8150-13f2694f9f42 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:17.782320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a093ff-6c03-4e40-807e-c4625dd599ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:17.783818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a093ff-6c03-4e40-807e-c4625dd599ab None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.784560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a093ff-6c03-4e40-807e-c4625dd599ab None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.785022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a093ff-6c03-4e40-807e-c4625dd599ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.794475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a093ff-6c03-4e40-807e-c4625dd599ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:17.795929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a093ff-6c03-4e40-807e-c4625dd599ab None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d0a093ff-6c03-4e40-807e-c4625dd599ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:17.812081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:17.812988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:17.813878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 246/495] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Jun 26 02:52:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:17.814936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b2e6c-e33b-403b-978c-82d020a67082 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.814936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b2e6c-e33b-403b-978c-82d020a67082 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.815218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b2e6c-e33b-403b-978c-82d020a67082 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.816579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 247/496] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:17.818747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b2e6c-e33b-403b-978c-82d020a67082 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:17.870733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823deacf-779c-4c78-8150-13f2694f9f42 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:17.904639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa9e17-4270-4fe4-bced-34af13cd1a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:17.905314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9e17-4270-4fe4-bced-34af13cd1a2f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.905584 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9e17-4270-4fe4-bced-34af13cd1a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.905860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa9e17-4270-4fe4-bced-34af13cd1a2f None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.906620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa9e17-4270-4fe4-bced-34af13cd1a2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:17.907222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa9e17-4270-4fe4-bced-34af13cd1a2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:17.908604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa9e17-4270-4fe4-bced-34af13cd1a2f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:17.915386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:17.917066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:17.920636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 250/497] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:17.921865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaadb3-4972-4a24-b60c-b8b86fded292 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.922557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaadb3-4972-4a24-b60c-b8b86fded292 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.923138 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b2e6c-e33b-403b-978c-82d020a67082 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:17.924813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eaadb3-4972-4a24-b60c-b8b86fded292 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.926131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 251/498] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:17.926860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eaadb3-4972-4a24-b60c-b8b86fded292 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:17.958890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f86878-a166-4e55-8f6a-0fc8619ceb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:17.960038 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86878-a166-4e55-8f6a-0fc8619ceb7c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.960318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86878-a166-4e55-8f6a-0fc8619ceb7c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.960588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f86878-a166-4e55-8f6a-0fc8619ceb7c None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.961343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f86878-a166-4e55-8f6a-0fc8619ceb7c None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:17.961637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f86878-a166-4e55-8f6a-0fc8619ceb7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:17.963047 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f86878-a166-4e55-8f6a-0fc8619ceb7c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:17.972073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 248/499] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:17.973056 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec31ae-36c5-48ec-bdff-f0415e9f0f76 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:17.973343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec31ae-36c5-48ec-bdff-f0415e9f0f76 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:17.973617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec31ae-36c5-48ec-bdff-f0415e9f0f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:17.989006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-efec31ae-36c5-48ec-bdff-f0415e9f0f76 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:17.990215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-efec31ae-36c5-48ec-bdff-f0415e9f0f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:18.040420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eaadb3-4972-4a24-b60c-b8b86fded292 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:18.057275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 249/500] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:18.058134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec31ae-36c5-48ec-bdff-f0415e9f0f76 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:18.083459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2b284-83cb-48c3-8e0c-e0883b72ade1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:18.084130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2b284-83cb-48c3-8e0c-e0883b72ade1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.084414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2b284-83cb-48c3-8e0c-e0883b72ade1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.084686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2b284-83cb-48c3-8e0c-e0883b72ade1 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.085536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2b284-83cb-48c3-8e0c-e0883b72ade1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:18.085835 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2b284-83cb-48c3-8e0c-e0883b72ade1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:18.087311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2b284-83cb-48c3-8e0c-e0883b72ade1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:18.122591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 252/501] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Jun 26 02:52:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:18.124141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcda0c-44f7-470c-b386-476046d34ddf None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.124433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcda0c-44f7-470c-b386-476046d34ddf None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.124710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dcda0c-44f7-470c-b386-476046d34ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.141580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-79dcda0c-44f7-470c-b386-476046d34ddf None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:18.142805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-79dcda0c-44f7-470c-b386-476046d34ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:18.147218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:18.160606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec31ae-36c5-48ec-bdff-f0415e9f0f76 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:18.193506 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7df7f-819c-44f4-8e95-ffe918f0f54d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:18.194320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7df7f-819c-44f4-8e95-ffe918f0f54d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.194571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:18.194689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7df7f-819c-44f4-8e95-ffe918f0f54d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.194857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc7df7f-819c-44f4-8e95-ffe918f0f54d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.202345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7df7f-819c-44f4-8e95-ffe918f0f54d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:18.203775 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc7df7f-819c-44f4-8e95-ffe918f0f54d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:18.209195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 253/502] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:18.209889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcda0c-44f7-470c-b386-476046d34ddf None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-afc7df7f-819c-44f4-8e95-ffe918f0f54d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:18.217812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:18.218705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:18.219595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 250/503] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:18.220213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7df7f-819c-44f4-8e95-ffe918f0f54d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:18.311608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dcda0c-44f7-470c-b386-476046d34ddf None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:18.323990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc7df7f-819c-44f4-8e95-ffe918f0f54d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:18.346296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc371e-901a-45e6-a42d-b434d5e5d33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:18.347083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc371e-901a-45e6-a42d-b434d5e5d33d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.347376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc371e-901a-45e6-a42d-b434d5e5d33d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.347653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfc371e-901a-45e6-a42d-b434d5e5d33d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.357425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc371e-901a-45e6-a42d-b434d5e5d33d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:18.357908 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf66a5f-a13c-465a-8503-e58dc57175a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:18.358717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf66a5f-a13c-465a-8503-e58dc57175a4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.358997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf66a5f-a13c-465a-8503-e58dc57175a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.359087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfc371e-901a-45e6-a42d-b434d5e5d33d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:18.359266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf66a5f-a13c-465a-8503-e58dc57175a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.368982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf66a5f-a13c-465a-8503-e58dc57175a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1cfc371e-901a-45e6-a42d-b434d5e5d33d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:18.377102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:18.378187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:18.379059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:18.379059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:18.379059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:18.379059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-4cf66a5f-a13c-465a-8503-e58dc57175a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:18.379273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:18.380055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:18.380817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:18.381413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 254/505] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:18.381476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 251/505] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:18.381476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf66a5f-a13c-465a-8503-e58dc57175a4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:18.381609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc371e-901a-45e6-a42d-b434d5e5d33d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:18.398137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:18.426439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:18.483825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfc371e-901a-45e6-a42d-b434d5e5d33d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:18.485238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf66a5f-a13c-465a-8503-e58dc57175a4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:18.517037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a189755-880a-4cc6-bf75-9502854766ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:18.517666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a189755-880a-4cc6-bf75-9502854766ee None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.517950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a189755-880a-4cc6-bf75-9502854766ee None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.518266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a189755-880a-4cc6-bf75-9502854766ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.519099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cf405-6b0b-40f5-af0d-7102033f07d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:18.519762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cf405-6b0b-40f5-af0d-7102033f07d5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.520048 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cf405-6b0b-40f5-af0d-7102033f07d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.520329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38cf405-6b0b-40f5-af0d-7102033f07d5 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.521167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cf405-6b0b-40f5-af0d-7102033f07d5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:18.521482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cf405-6b0b-40f5-af0d-7102033f07d5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:18.522947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38cf405-6b0b-40f5-af0d-7102033f07d5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:18.559545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 252/506] 10.222.0.13 () {56 vars in 1137 bytes} [Sun Jun 26 02:52:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:18.560461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cf405-6b0b-40f5-af0d-7102033f07d5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:18.586172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a189755-880a-4cc6-bf75-9502854766ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:18.587620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a189755-880a-4cc6-bf75-9502854766ee None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:18.630238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:18.630741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:18.668895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38cf405-6b0b-40f5-af0d-7102033f07d5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:18.689189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 255/507] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:18.690019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a189755-880a-4cc6-bf75-9502854766ee None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:18.702283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbc0ad-62ff-4729-9bce-ee42e2abeee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:18.703077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbc0ad-62ff-4729-9bce-ee42e2abeee1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.703363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbc0ad-62ff-4729-9bce-ee42e2abeee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.703700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbc0ad-62ff-4729-9bce-ee42e2abeee1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.713631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbc0ad-62ff-4729-9bce-ee42e2abeee1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:18.715774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbc0ad-62ff-4729-9bce-ee42e2abeee1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a9fbc0ad-62ff-4729-9bce-ee42e2abeee1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:18.733995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:18.735008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:18.735986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:18.735986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:18.735986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:18.735986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:18.735986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 253/508] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:18.735986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbc0ad-62ff-4729-9bce-ee42e2abeee1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:18.790269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a189755-880a-4cc6-bf75-9502854766ee None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:18.822081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407077c-09dc-4ec6-9e04-e439b143b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:18.822730 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407077c-09dc-4ec6-9e04-e439b143b18e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.823011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407077c-09dc-4ec6-9e04-e439b143b18e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.823809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407077c-09dc-4ec6-9e04-e439b143b18e None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.824523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407077c-09dc-4ec6-9e04-e439b143b18e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:18.824825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407077c-09dc-4ec6-9e04-e439b143b18e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:18.826195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407077c-09dc-4ec6-9e04-e439b143b18e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:18.833266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbc0ad-62ff-4729-9bce-ee42e2abeee1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:18.833998 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:18.837893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 256/509] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:18.839618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407077c-09dc-4ec6-9e04-e439b143b18e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:18.851886 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:18.869152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b1689-9032-4af5-8db8-12e27224c17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:18.870117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b1689-9032-4af5-8db8-12e27224c17f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.870557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b1689-9032-4af5-8db8-12e27224c17f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.870836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b1689-9032-4af5-8db8-12e27224c17f None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.871587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b1689-9032-4af5-8db8-12e27224c17f None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:18.871899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b1689-9032-4af5-8db8-12e27224c17f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:18.873287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b1689-9032-4af5-8db8-12e27224c17f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:18.881604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 254/510] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:18.882430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b1689-9032-4af5-8db8-12e27224c17f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:18.942759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407077c-09dc-4ec6-9e04-e439b143b18e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:18.975043 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cc9cf-6e25-4597-8dbb-2c6756d0fc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:18.975682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cc9cf-6e25-4597-8dbb-2c6756d0fc92 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:18.975964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cc9cf-6e25-4597-8dbb-2c6756d0fc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:18.976240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0cc9cf-6e25-4597-8dbb-2c6756d0fc92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:18.981021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b1689-9032-4af5-8db8-12e27224c17f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.014851 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e3450-a0c1-4b51-bfbb-1581eb53fe58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.015642 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e3450-a0c1-4b51-bfbb-1581eb53fe58 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.015930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e3450-a0c1-4b51-bfbb-1581eb53fe58 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.016208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150e3450-a0c1-4b51-bfbb-1581eb53fe58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.026776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e3450-a0c1-4b51-bfbb-1581eb53fe58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.028212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150e3450-a0c1-4b51-bfbb-1581eb53fe58 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.041033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:19.043610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0cc9cf-6e25-4597-8dbb-2c6756d0fc92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-150e3450-a0c1-4b51-bfbb-1581eb53fe58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:19.044745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:19.045648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:19.046513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:19.046513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:19.046513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:19.046513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:19.046716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0cc9cf-6e25-4597-8dbb-2c6756d0fc92 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.046789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 255/511] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:19.046789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e3450-a0c1-4b51-bfbb-1581eb53fe58 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.077860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:19.145393 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150e3450-a0c1-4b51-bfbb-1581eb53fe58 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.148170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 257/512] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:19.149083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cc9cf-6e25-4597-8dbb-2c6756d0fc92 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.178380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f237c53-358d-4add-a3ee-3421803ca347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.179044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f237c53-358d-4add-a3ee-3421803ca347 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.179329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f237c53-358d-4add-a3ee-3421803ca347 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.179606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f237c53-358d-4add-a3ee-3421803ca347 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.180329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f237c53-358d-4add-a3ee-3421803ca347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:19.180644 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f237c53-358d-4add-a3ee-3421803ca347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.182020 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f237c53-358d-4add-a3ee-3421803ca347 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.193934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 256/513] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:19.194713 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f237c53-358d-4add-a3ee-3421803ca347 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.242488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:19.246154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0cc9cf-6e25-4597-8dbb-2c6756d0fc92 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.279462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0cfe5-1ad9-46b8-9d8f-78fb1b378786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.280242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0cfe5-1ad9-46b8-9d8f-78fb1b378786 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.280522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0cfe5-1ad9-46b8-9d8f-78fb1b378786 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.280794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c0cfe5-1ad9-46b8-9d8f-78fb1b378786 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.284841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:19.287972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0cfe5-1ad9-46b8-9d8f-78fb1b378786 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.289390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c0cfe5-1ad9-46b8-9d8f-78fb1b378786 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.294311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f237c53-358d-4add-a3ee-3421803ca347 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-65c0cfe5-1ad9-46b8-9d8f-78fb1b378786 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:19.305225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:19.306309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 258/514] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:19.312467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0cfe5-1ad9-46b8-9d8f-78fb1b378786 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.330168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f463-156f-4269-a340-cee5620ab102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.330957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f463-156f-4269-a340-cee5620ab102 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.331240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f463-156f-4269-a340-cee5620ab102 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.331514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda1f463-156f-4269-a340-cee5620ab102 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.341935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1f463-156f-4269-a340-cee5620ab102 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.343432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda1f463-156f-4269-a340-cee5620ab102 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-cda1f463-156f-4269-a340-cee5620ab102 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:19.360415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:19.361465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:19.362771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:19.362771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:19.362771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:19.362771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:19.362771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 257/515] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:19.362771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f463-156f-4269-a340-cee5620ab102 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.408023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c0cfe5-1ad9-46b8-9d8f-78fb1b378786 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.441401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8636d-a388-4b80-a497-fbec2862459a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.442071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8636d-a388-4b80-a497-fbec2862459a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.442368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8636d-a388-4b80-a497-fbec2862459a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.442655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b8636d-a388-4b80-a497-fbec2862459a None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.443379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8636d-a388-4b80-a497-fbec2862459a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:19.443692 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8636d-a388-4b80-a497-fbec2862459a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.445067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b8636d-a388-4b80-a497-fbec2862459a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.456664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 259/516] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:19.457428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8636d-a388-4b80-a497-fbec2862459a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.465427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda1f463-156f-4269-a340-cee5620ab102 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.478771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:19.499190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d283d3e-ec14-4790-9449-ebe34bbda19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.500003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d283d3e-ec14-4790-9449-ebe34bbda19c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.500292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d283d3e-ec14-4790-9449-ebe34bbda19c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.500569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d283d3e-ec14-4790-9449-ebe34bbda19c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.510429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d283d3e-ec14-4790-9449-ebe34bbda19c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.512401 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d283d3e-ec14-4790-9449-ebe34bbda19c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.515144 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-4d283d3e-ec14-4790-9449-ebe34bbda19c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:19.530857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:19.532140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:19.533511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:19.533511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:19.533511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:19.533511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:19.533511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 258/517] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:19.533511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d283d3e-ec14-4790-9449-ebe34bbda19c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.556904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b8636d-a388-4b80-a497-fbec2862459a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.591238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50691f-f45f-4a18-9d1b-d5899bd0921b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.591903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50691f-f45f-4a18-9d1b-d5899bd0921b None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.592185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50691f-f45f-4a18-9d1b-d5899bd0921b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.592458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50691f-f45f-4a18-9d1b-d5899bd0921b None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.593161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50691f-f45f-4a18-9d1b-d5899bd0921b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.594621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50691f-f45f-4a18-9d1b-d5899bd0921b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.613373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 260/518] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:52:19 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:19.614414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50691f-f45f-4a18-9d1b-d5899bd0921b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.632670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d283d3e-ec14-4790-9449-ebe34bbda19c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.674650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb09658-67fe-475c-8144-be3cab123da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.675354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb09658-67fe-475c-8144-be3cab123da4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.675703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb09658-67fe-475c-8144-be3cab123da4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.676045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb09658-67fe-475c-8144-be3cab123da4 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.676989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb09658-67fe-475c-8144-be3cab123da4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:19.677365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb09658-67fe-475c-8144-be3cab123da4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.678943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb09658-67fe-475c-8144-be3cab123da4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.704716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 259/519] 10.222.0.13 () {56 vars in 1137 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:19.705553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb09658-67fe-475c-8144-be3cab123da4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.709112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:19.736766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50691f-f45f-4a18-9d1b-d5899bd0921b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.769997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:19.776452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f661b-d955-494d-8da4-afd7ed17632d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.777125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f661b-d955-494d-8da4-afd7ed17632d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.777412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f661b-d955-494d-8da4-afd7ed17632d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.777689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f661b-d955-494d-8da4-afd7ed17632d None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.778578 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f661b-d955-494d-8da4-afd7ed17632d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:19.778856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f661b-d955-494d-8da4-afd7ed17632d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.780229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f661b-d955-494d-8da4-afd7ed17632d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.794091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 261/520] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:19.794340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f661b-d955-494d-8da4-afd7ed17632d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.826457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb09658-67fe-475c-8144-be3cab123da4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.862831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6e177-249a-4ad6-9765-36b89e377106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.863894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6e177-249a-4ad6-9765-36b89e377106 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.864178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6e177-249a-4ad6-9765-36b89e377106 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.864454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb6e177-249a-4ad6-9765-36b89e377106 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.872039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6e177-249a-4ad6-9765-36b89e377106 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.874279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb6e177-249a-4ad6-9765-36b89e377106 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-4cb6e177-249a-4ad6-9765-36b89e377106 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:19.890577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:19.891429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:19.892270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:19.892270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:19.892270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:19.892270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:19.892270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:19.892270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:19.892270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:19.892270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:19.892270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 260/521] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:19.892740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6e177-249a-4ad6-9765-36b89e377106 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:19.914389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f661b-d955-494d-8da4-afd7ed17632d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:19.951422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fb1e6-baef-44dd-a20b-4a3359376a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:19.952008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fb1e6-baef-44dd-a20b-4a3359376a91 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:19.952299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fb1e6-baef-44dd-a20b-4a3359376a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:19.952581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4fb1e6-baef-44dd-a20b-4a3359376a91 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:19.953309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4fb1e6-baef-44dd-a20b-4a3359376a91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:19.953623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4fb1e6-baef-44dd-a20b-4a3359376a91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:19.955027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4fb1e6-baef-44dd-a20b-4a3359376a91 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:19.957415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:19.972825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 262/522] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:19.972825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fb1e6-baef-44dd-a20b-4a3359376a91 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:20.004288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb6e177-249a-4ad6-9765-36b89e377106 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:20.019230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:20.041154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebe2df-e943-40ac-9f3e-ce30ece9b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:20.041891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebe2df-e943-40ac-9f3e-ce30ece9b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:20.042729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebe2df-e943-40ac-9f3e-ce30ece9b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:20.042980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ebe2df-e943-40ac-9f3e-ce30ece9b6f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:20.075183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4fb1e6-baef-44dd-a20b-4a3359376a91 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:20.110659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f07181-432e-497c-8203-ccac7c109085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:20.111636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f07181-432e-497c-8203-ccac7c109085 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:20.112030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f07181-432e-497c-8203-ccac7c109085 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:20.112362 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f07181-432e-497c-8203-ccac7c109085 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:20.113042 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebe2df-e943-40ac-9f3e-ce30ece9b6f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:20.114637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ebe2df-e943-40ac-9f3e-ce30ece9b6f2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:20.122479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f07181-432e-497c-8203-ccac7c109085 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:20.124298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f07181-432e-497c-8203-ccac7c109085 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-61f07181-432e-497c-8203-ccac7c109085 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:20.138628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:20.139713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:20.140921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:20.140921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:20.140921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:20.140921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:20.140921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:20.140921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:20.140921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:20.140921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:20.140921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 26 02:52:20.140921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:20.141792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 263/523] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:20.142610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f07181-432e-497c-8203-ccac7c109085 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:20.204192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:20.228697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 261/524] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:20.228697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebe2df-e943-40ac-9f3e-ce30ece9b6f2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:20.241052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:20.256060 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f07181-432e-497c-8203-ccac7c109085 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:20.290476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36624d5f-0888-46f4-9638-937318bef511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:20.291202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36624d5f-0888-46f4-9638-937318bef511 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:20.291421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36624d5f-0888-46f4-9638-937318bef511 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:20.291701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36624d5f-0888-46f4-9638-937318bef511 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:20.293021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36624d5f-0888-46f4-9638-937318bef511 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:20.293338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36624d5f-0888-46f4-9638-937318bef511 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:20.294798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36624d5f-0888-46f4-9638-937318bef511 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:20.318574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 264/525] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:20.319447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36624d5f-0888-46f4-9638-937318bef511 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:20.329575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ebe2df-e943-40ac-9f3e-ce30ece9b6f2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:20.364686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36d2d8-32ad-4085-a3f2-d8e67160ca21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:20.365563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36d2d8-32ad-4085-a3f2-d8e67160ca21 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:20.365910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36d2d8-32ad-4085-a3f2-d8e67160ca21 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:20.366278 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36d2d8-32ad-4085-a3f2-d8e67160ca21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:20.372925 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36d2d8-32ad-4085-a3f2-d8e67160ca21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:20.374649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36d2d8-32ad-4085-a3f2-d8e67160ca21 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7e36d2d8-32ad-4085-a3f2-d8e67160ca21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:20.388970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:20.390018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:20.391014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:20.391014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:20.391014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:20.391014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:20.391014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:20.391014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:20.391014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:20.391014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:20.391014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 262/526] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:20.391470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36d2d8-32ad-4085-a3f2-d8e67160ca21 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:20.424527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36624d5f-0888-46f4-9638-937318bef511 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:20.441635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:20.462455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cc33f-506e-4b40-96e4-b159f75e7014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:20.463136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cc33f-506e-4b40-96e4-b159f75e7014 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:20.463500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cc33f-506e-4b40-96e4-b159f75e7014 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:20.463869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617cc33f-506e-4b40-96e4-b159f75e7014 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:20.464775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cc33f-506e-4b40-96e4-b159f75e7014 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:20.465157 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cc33f-506e-4b40-96e4-b159f75e7014 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:20.466283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:20.467377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617cc33f-506e-4b40-96e4-b159f75e7014 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:20.486287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 265/527] 10.222.0.13 () {56 vars in 1139 bytes} [Sun Jun 26 02:52:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:20.487204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cc33f-506e-4b40-96e4-b159f75e7014 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:20.498990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36d2d8-32ad-4085-a3f2-d8e67160ca21 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:20.534533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee6c5d-9dae-4416-a102-f8041913cbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:20.535321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee6c5d-9dae-4416-a102-f8041913cbca None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:20.535608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee6c5d-9dae-4416-a102-f8041913cbca None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:20.535885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dee6c5d-9dae-4416-a102-f8041913cbca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:20.546585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee6c5d-9dae-4416-a102-f8041913cbca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:20.548033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dee6c5d-9dae-4416-a102-f8041913cbca None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7dee6c5d-9dae-4416-a102-f8041913cbca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:20.566585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:20.567421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:20.568327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:20.568327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:20.568327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:20.568327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:20.568327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 263/528] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:20.569176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee6c5d-9dae-4416-a102-f8041913cbca None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:20.597482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617cc33f-506e-4b40-96e4-b159f75e7014 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:20.631614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68217e6-96a9-4ceb-ab91-0b8d2f2d0a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:20.632261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68217e6-96a9-4ceb-ab91-0b8d2f2d0a39 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:20.632547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68217e6-96a9-4ceb-ab91-0b8d2f2d0a39 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:20.632825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68217e6-96a9-4ceb-ab91-0b8d2f2d0a39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:20.633645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68217e6-96a9-4ceb-ab91-0b8d2f2d0a39 None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=9ee0150dfb3f42709029e42d0fd9105d) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:20.634240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68217e6-96a9-4ceb-ab91-0b8d2f2d0a39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:20.636269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68217e6-96a9-4ceb-ab91-0b8d2f2d0a39 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:20.650307 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 266/529] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:20 2022] GET /identity/v3/role_assignments?user.id=9ee0150dfb3f42709029e42d0fd9105d&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:20.650307 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68217e6-96a9-4ceb-ab91-0b8d2f2d0a39 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:20.694925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:20.699188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dee6c5d-9dae-4416-a102-f8041913cbca None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:20.732818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:20.744708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97768ccd-1c1a-49d5-8cd2-63b4b778f89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:20.746217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97768ccd-1c1a-49d5-8cd2-63b4b778f89a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:20.746645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97768ccd-1c1a-49d5-8cd2-63b4b778f89a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:20.747062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97768ccd-1c1a-49d5-8cd2-63b4b778f89a None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:20.748027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97768ccd-1c1a-49d5-8cd2-63b4b778f89a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:20.748526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97768ccd-1c1a-49d5-8cd2-63b4b778f89a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:20.750790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97768ccd-1c1a-49d5-8cd2-63b4b778f89a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:20.772525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 264/530] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:20.773632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97768ccd-1c1a-49d5-8cd2-63b4b778f89a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:20.784205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68217e6-96a9-4ceb-ab91-0b8d2f2d0a39 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:20.830324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4652e10-1420-4f77-97c4-fd5d3820df98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:20.830989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4652e10-1420-4f77-97c4-fd5d3820df98 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:20.831525 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4652e10-1420-4f77-97c4-fd5d3820df98 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:20.831816 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4652e10-1420-4f77-97c4-fd5d3820df98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:20.892905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97768ccd-1c1a-49d5-8cd2-63b4b778f89a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:20.929354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6518-1a91-4c55-961f-fcedcfe4e968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:20.930141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6518-1a91-4c55-961f-fcedcfe4e968 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:20.930400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6518-1a91-4c55-961f-fcedcfe4e968 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:20.930689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116b6518-1a91-4c55-961f-fcedcfe4e968 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:20.931570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6518-1a91-4c55-961f-fcedcfe4e968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:20.931877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6518-1a91-4c55-961f-fcedcfe4e968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:20.934484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116b6518-1a91-4c55-961f-fcedcfe4e968 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:20.940496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4652e10-1420-4f77-97c4-fd5d3820df98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:20.942611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4652e10-1420-4f77-97c4-fd5d3820df98 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:20.956072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 265/531] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:20.956072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6518-1a91-4c55-961f-fcedcfe4e968 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:20.987824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:21.010508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:21.059256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 267/532] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:21.059256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4652e10-1420-4f77-97c4-fd5d3820df98 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:21.075569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116b6518-1a91-4c55-961f-fcedcfe4e968 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:21.112868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2542607-9639-4a87-9e73-827fed7b27ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:21.113936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2542607-9639-4a87-9e73-827fed7b27ec None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:21.114462 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2542607-9639-4a87-9e73-827fed7b27ec None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:21.114936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2542607-9639-4a87-9e73-827fed7b27ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:21.116123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2542607-9639-4a87-9e73-827fed7b27ec None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=0dc5b601421a41d1a30a10e86967a920) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:21.116686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2542607-9639-4a87-9e73-827fed7b27ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:21.118493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2542607-9639-4a87-9e73-827fed7b27ec None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:21.134379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 266/533] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:20 2022] GET /identity/v3/role_assignments?user.id=0dc5b601421a41d1a30a10e86967a920&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:21.135714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2542607-9639-4a87-9e73-827fed7b27ec None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:21.166611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4652e10-1420-4f77-97c4-fd5d3820df98 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:21.208793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e7356-7571-43e9-8966-3223612f45b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:21.209605 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e7356-7571-43e9-8966-3223612f45b4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:21.209893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e7356-7571-43e9-8966-3223612f45b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:21.210202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146e7356-7571-43e9-8966-3223612f45b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:21.225182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e7356-7571-43e9-8966-3223612f45b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:21.235693 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146e7356-7571-43e9-8966-3223612f45b4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:21.250592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-146e7356-7571-43e9-8966-3223612f45b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:21.267798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:21.268910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:21.269848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:21.269848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:21.269848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:21.269848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:21.269848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:21.269848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:21.269848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:21.269848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:21.269848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 268/534] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:21.277384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e7356-7571-43e9-8966-3223612f45b4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:21.289900 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2542607-9639-4a87-9e73-827fed7b27ec None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:21.301981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:21.329209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221f17d-fdf3-49c9-a4be-5de296b575ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:21.331283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221f17d-fdf3-49c9-a4be-5de296b575ef None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:21.331742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221f17d-fdf3-49c9-a4be-5de296b575ef None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:21.332912 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8221f17d-fdf3-49c9-a4be-5de296b575ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:21.358380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221f17d-fdf3-49c9-a4be-5de296b575ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:21.362231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8221f17d-fdf3-49c9-a4be-5de296b575ef None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8221f17d-fdf3-49c9-a4be-5de296b575ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:21.394741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:21.395799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:21.396829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:21.396829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:21.396829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:21.396829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:21.396829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 267/535] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:21.400387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221f17d-fdf3-49c9-a4be-5de296b575ef None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:21.436429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146e7356-7571-43e9-8966-3223612f45b4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:21.471751 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483910fe-2a33-4a5f-b698-1695d919f620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:21.472708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483910fe-2a33-4a5f-b698-1695d919f620 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:21.473132 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483910fe-2a33-4a5f-b698-1695d919f620 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:21.473546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483910fe-2a33-4a5f-b698-1695d919f620 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:21.474503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483910fe-2a33-4a5f-b698-1695d919f620 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:21.474952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483910fe-2a33-4a5f-b698-1695d919f620 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:21.476567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483910fe-2a33-4a5f-b698-1695d919f620 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:21.490302 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 269/536] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:21.498150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483910fe-2a33-4a5f-b698-1695d919f620 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:21.568084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8221f17d-fdf3-49c9-a4be-5de296b575ef None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:21.607783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:21.636176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561b7f-e6d0-4f93-a975-7aebd9c0b729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:21.637215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561b7f-e6d0-4f93-a975-7aebd9c0b729 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:21.637695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561b7f-e6d0-4f93-a975-7aebd9c0b729 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:21.638198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd561b7f-e6d0-4f93-a975-7aebd9c0b729 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:21.639121 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd561b7f-e6d0-4f93-a975-7aebd9c0b729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:21.639643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd561b7f-e6d0-4f93-a975-7aebd9c0b729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:21.642000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd561b7f-e6d0-4f93-a975-7aebd9c0b729 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:21.670649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 268/537] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:21.683049 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561b7f-e6d0-4f93-a975-7aebd9c0b729 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:21.683177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483910fe-2a33-4a5f-b698-1695d919f620 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:21.719704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:21.724450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91ae3c-d11d-48ce-bf9b-42a8f1d79376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:21.725280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91ae3c-d11d-48ce-bf9b-42a8f1d79376 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:21.725568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91ae3c-d11d-48ce-bf9b-42a8f1d79376 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:21.726189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91ae3c-d11d-48ce-bf9b-42a8f1d79376 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:21.727339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91ae3c-d11d-48ce-bf9b-42a8f1d79376 None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=5e65f36f772b4220a909f8fda83ba42e) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:21.727724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91ae3c-d11d-48ce-bf9b-42a8f1d79376 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:21.729383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee91ae3c-d11d-48ce-bf9b-42a8f1d79376 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:21.739372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 270/538] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:21 2022] GET /identity/v3/role_assignments?user.id=5e65f36f772b4220a909f8fda83ba42e&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 259 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:21.741326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9466e65-a7a4-45cb-9e52-fec89205fa0e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:21.741635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9466e65-a7a4-45cb-9e52-fec89205fa0e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:21.741922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9466e65-a7a4-45cb-9e52-fec89205fa0e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:21.744238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 271/539] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:21.751895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08680-7839-4c38-be53-eb5ce9647b11 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:21.752187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08680-7839-4c38-be53-eb5ce9647b11 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:21.752471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb08680-7839-4c38-be53-eb5ce9647b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:21.787260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4fb08680-7839-4c38-be53-eb5ce9647b11 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:21.788965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4fb08680-7839-4c38-be53-eb5ce9647b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:21.816228 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd561b7f-e6d0-4f93-a975-7aebd9c0b729 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:21.860347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed1dc2-a1b5-4b16-9dd4-c6e6df311c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:21.861746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed1dc2-a1b5-4b16-9dd4-c6e6df311c4d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:21.862033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed1dc2-a1b5-4b16-9dd4-c6e6df311c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:21.862335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ed1dc2-a1b5-4b16-9dd4-c6e6df311c4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:21.880252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed1dc2-a1b5-4b16-9dd4-c6e6df311c4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:21.881784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ed1dc2-a1b5-4b16-9dd4-c6e6df311c4d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-36ed1dc2-a1b5-4b16-9dd4-c6e6df311c4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:21.899970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:21.900870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:21.902262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:21.902262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:21.902262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:21.902262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:21.902262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:21.902262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:21.902262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:21.902262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:21.902262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 269/540] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:21.908285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed1dc2-a1b5-4b16-9dd4-c6e6df311c4d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:21.942616 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 272/541] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:21.963639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08680-7839-4c38-be53-eb5ce9647b11 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:22.030297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:22.073527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb08680-7839-4c38-be53-eb5ce9647b11 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:22.110079 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:22.113791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fa43f-cfe4-4ac8-892e-d17c64312ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:22.114928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fa43f-cfe4-4ac8-892e-d17c64312ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:22.116256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fa43f-cfe4-4ac8-892e-d17c64312ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:22.116706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fa43f-cfe4-4ac8-892e-d17c64312ff6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:22.128600 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fa43f-cfe4-4ac8-892e-d17c64312ff6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-fd8fa43f-cfe4-4ac8-892e-d17c64312ff6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:22.136830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:22.137879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:22.138895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:22.146216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 273/542] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:22.147724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fa43f-cfe4-4ac8-892e-d17c64312ff6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:22.184054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ed1dc2-a1b5-4b16-9dd4-c6e6df311c4d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:22.256037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fa43f-cfe4-4ac8-892e-d17c64312ff6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:22.296356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9639e-1892-4f93-ae6a-e4b355ec3058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:22.297029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9639e-1892-4f93-ae6a-e4b355ec3058 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:22.297319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9639e-1892-4f93-ae6a-e4b355ec3058 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:22.297601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b9639e-1892-4f93-ae6a-e4b355ec3058 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:22.298430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9639e-1892-4f93-ae6a-e4b355ec3058 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:22.298751 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9639e-1892-4f93-ae6a-e4b355ec3058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:22.299420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:22.300724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b9639e-1892-4f93-ae6a-e4b355ec3058 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:22.310415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 274/543] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:22.310749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f807b-c1ba-4031-95d0-a8b50a6bf6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:22.311421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f807b-c1ba-4031-95d0-a8b50a6bf6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:22.311708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f807b-c1ba-4031-95d0-a8b50a6bf6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:22.311988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6f807b-c1ba-4031-95d0-a8b50a6bf6e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:22.312787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f807b-c1ba-4031-95d0-a8b50a6bf6e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:22.313102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f807b-c1ba-4031-95d0-a8b50a6bf6e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:22.318341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6f807b-c1ba-4031-95d0-a8b50a6bf6e5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:22.318460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9639e-1892-4f93-ae6a-e4b355ec3058 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:22.342614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 270/544] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 434 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:22.350063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f807b-c1ba-4031-95d0-a8b50a6bf6e5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:22.462358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b9639e-1892-4f93-ae6a-e4b355ec3058 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:22.500324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b03da-0700-4812-b746-65e47dd4f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:22.501128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b03da-0700-4812-b746-65e47dd4f023 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:22.501423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b03da-0700-4812-b746-65e47dd4f023 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:22.501708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b03da-0700-4812-b746-65e47dd4f023 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:22.513872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b03da-0700-4812-b746-65e47dd4f023 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:22.515584 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b03da-0700-4812-b746-65e47dd4f023 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0c1b03da-0700-4812-b746-65e47dd4f023 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:22.535330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:22.536234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:22.537054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:22.537054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:22.537054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:22.537054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:22.537724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 275/545] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:22.544437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b03da-0700-4812-b746-65e47dd4f023 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:22.585766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6f807b-c1ba-4031-95d0-a8b50a6bf6e5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:22.618406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:22.625547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd9fc4-35a7-49e2-92f8-bd24e03239c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:22.626557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd9fc4-35a7-49e2-92f8-bd24e03239c7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:22.627041 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd9fc4-35a7-49e2-92f8-bd24e03239c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:22.627510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd9fc4-35a7-49e2-92f8-bd24e03239c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:22.628533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd9fc4-35a7-49e2-92f8-bd24e03239c7 None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=8ff708d6758d4f7b913e763824f3b791) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:22.629061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd9fc4-35a7-49e2-92f8-bd24e03239c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:22.630733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd9fc4-35a7-49e2-92f8-bd24e03239c7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:22.637787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 271/546] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:22 2022] GET /identity/v3/role_assignments?user.id=8ff708d6758d4f7b913e763824f3b791&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 259 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:22.721310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b03da-0700-4812-b746-65e47dd4f023 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:22.765858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb23cb50-6d6d-4395-a7ec-2ddc411b4d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:22.766577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23cb50-6d6d-4395-a7ec-2ddc411b4d07 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:22.766862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23cb50-6d6d-4395-a7ec-2ddc411b4d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:22.767148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb23cb50-6d6d-4395-a7ec-2ddc411b4d07 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:22.767895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23cb50-6d6d-4395-a7ec-2ddc411b4d07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:22.768206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23cb50-6d6d-4395-a7ec-2ddc411b4d07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:22.769611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb23cb50-6d6d-4395-a7ec-2ddc411b4d07 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:22.790444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 276/547] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:22.791540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd9fc4-35a7-49e2-92f8-bd24e03239c7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:22.808038 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:22.925932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd9fc4-35a7-49e2-92f8-bd24e03239c7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:22.952595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2201871-1d05-46a5-8cb6-74b66c177d06 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:22.952880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2201871-1d05-46a5-8cb6-74b66c177d06 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:22.953167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2201871-1d05-46a5-8cb6-74b66c177d06 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:22.956058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 277/548] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:22.960199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679978b-fb10-443f-8a9e-70e3eec0bb11 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:22.960484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679978b-fb10-443f-8a9e-70e3eec0bb11 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:22.960765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5679978b-fb10-443f-8a9e-70e3eec0bb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:22.962950 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e56519-f67d-4d0b-821a-72475ebe4b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:22.964515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e56519-f67d-4d0b-821a-72475ebe4b78 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:22.965028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e56519-f67d-4d0b-821a-72475ebe4b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:22.965424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e56519-f67d-4d0b-821a-72475ebe4b78 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:22.982871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e56519-f67d-4d0b-821a-72475ebe4b78 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:22.984683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e56519-f67d-4d0b-821a-72475ebe4b78 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:22.990976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5679978b-fb10-443f-8a9e-70e3eec0bb11 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:22.993528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5679978b-fb10-443f-8a9e-70e3eec0bb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b0e56519-f67d-4d0b-821a-72475ebe4b78 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:23.008680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:23.009988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:23.011218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:23.011218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:23.011218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:23.011218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:23.011218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:23.011218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:23.011218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:23.011218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:23.011218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:23.011218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:23.019908 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 272/549] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:23.023222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e56519-f67d-4d0b-821a-72475ebe4b78 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:23.131216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 278/550] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:23.146849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:23.192784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679978b-fb10-443f-8a9e-70e3eec0bb11 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:23.204611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e56519-f67d-4d0b-821a-72475ebe4b78 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:23.218097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:23.270669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86740096-f64b-447f-8987-21a08defcc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:23.271687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86740096-f64b-447f-8987-21a08defcc05 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:23.272171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86740096-f64b-447f-8987-21a08defcc05 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:23.272643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86740096-f64b-447f-8987-21a08defcc05 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:23.273692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86740096-f64b-447f-8987-21a08defcc05 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:23.275029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86740096-f64b-447f-8987-21a08defcc05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:23.276689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86740096-f64b-447f-8987-21a08defcc05 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:23.303539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 273/551] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Jun 26 02:52:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:23.309830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86740096-f64b-447f-8987-21a08defcc05 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:23.376133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5679978b-fb10-443f-8a9e-70e3eec0bb11 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:23.442613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c41c9f-dce8-4763-baf8-288ddeb0f33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:23.443695 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c41c9f-dce8-4763-baf8-288ddeb0f33c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:23.443983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c41c9f-dce8-4763-baf8-288ddeb0f33c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:23.444257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c41c9f-dce8-4763-baf8-288ddeb0f33c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:23.459910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86740096-f64b-447f-8987-21a08defcc05 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:23.465465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c41c9f-dce8-4763-baf8-288ddeb0f33c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-42c41c9f-dce8-4763-baf8-288ddeb0f33c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:23.478635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:23.479582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:23.480547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:23.490373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 279/552] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:23.495794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c41c9f-dce8-4763-baf8-288ddeb0f33c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:23.518117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:23.534978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b66e0-01a5-4ff6-9e38-280ba5ffc9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:23.535549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b66e0-01a5-4ff6-9e38-280ba5ffc9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:23.535830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b66e0-01a5-4ff6-9e38-280ba5ffc9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:23.536105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b66e0-01a5-4ff6-9e38-280ba5ffc9b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:23.625420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b66e0-01a5-4ff6-9e38-280ba5ffc9b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:23.627342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b66e0-01a5-4ff6-9e38-280ba5ffc9b1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:23.669852 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:23.701021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c41c9f-dce8-4763-baf8-288ddeb0f33c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:23.736324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2d4cfd-205e-497c-8397-6da1701b2409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:23.736995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d4cfd-205e-497c-8397-6da1701b2409 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:23.737277 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d4cfd-205e-497c-8397-6da1701b2409 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:23.737555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2d4cfd-205e-497c-8397-6da1701b2409 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:23.738330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d4cfd-205e-497c-8397-6da1701b2409 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:23.738637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d4cfd-205e-497c-8397-6da1701b2409 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:23.740014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2d4cfd-205e-497c-8397-6da1701b2409 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:23.753392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 280/553] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:23.754884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1d77e-22fd-4a43-b968-f7e217646442 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:23.755181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1d77e-22fd-4a43-b968-f7e217646442 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:23.755458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a1d77e-22fd-4a43-b968-f7e217646442 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:23.757307 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 281/554] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:23.758234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1d77e-22fd-4a43-b968-f7e217646442 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:23.764558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 274/555] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:23.765790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6cf49-c105-441c-b29a-6784f4310a59 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:23.766109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6cf49-c105-441c-b29a-6784f4310a59 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:23.766443 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6cf49-c105-441c-b29a-6784f4310a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:23.787028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-26e6cf49-c105-441c-b29a-6784f4310a59 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:23.788890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-26e6cf49-c105-441c-b29a-6784f4310a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:23.813126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:23.889261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a1d77e-22fd-4a43-b968-f7e217646442 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:23.889620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 275/556] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:23.890850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6cf49-c105-441c-b29a-6784f4310a59 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:23.923955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578fe83-753f-4c1b-aa96-3e807cb7c534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:23.924725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578fe83-753f-4c1b-aa96-3e807cb7c534 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:23.925018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578fe83-753f-4c1b-aa96-3e807cb7c534 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:23.925282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6578fe83-753f-4c1b-aa96-3e807cb7c534 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:23.933638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:23.939402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578fe83-753f-4c1b-aa96-3e807cb7c534 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:23.941685 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6578fe83-753f-4c1b-aa96-3e807cb7c534 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-6578fe83-753f-4c1b-aa96-3e807cb7c534 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:23.962054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:23.962962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:23.964349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:23.964349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:23.964349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:23.964349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:23.964349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 282/557] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:23.965054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578fe83-753f-4c1b-aa96-3e807cb7c534 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.033302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e6cf49-c105-441c-b29a-6784f4310a59 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.070439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c77f9-59be-44f4-84bb-fc63590df9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.071161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c77f9-59be-44f4-84bb-fc63590df9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.071439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c77f9-59be-44f4-84bb-fc63590df9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.071717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c77f9-59be-44f4-84bb-fc63590df9b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.078725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6578fe83-753f-4c1b-aa96-3e807cb7c534 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.083927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c77f9-59be-44f4-84bb-fc63590df9b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:24.085403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c77f9-59be-44f4-84bb-fc63590df9b6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-807c77f9-59be-44f4-84bb-fc63590df9b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:24.102981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:24.104336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:24.105628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:24.105628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:24.105628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:24.105628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:24.105628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 276/558] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:24.105628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c77f9-59be-44f4-84bb-fc63590df9b6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.114312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:24.119959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0ef1d4-93d0-4edb-bf67-a15807559523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.120754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ef1d4-93d0-4edb-bf67-a15807559523 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.121040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ef1d4-93d0-4edb-bf67-a15807559523 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.121315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0ef1d4-93d0-4edb-bf67-a15807559523 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.131975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0ef1d4-93d0-4edb-bf67-a15807559523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5b0ef1d4-93d0-4edb-bf67-a15807559523 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:24.140133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:24.141014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:24.141867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:24.146238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 283/559] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:24.147584 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee7f1-da4f-4b8e-b225-330b69323851 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.147876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee7f1-da4f-4b8e-b225-330b69323851 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.148157 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ee7f1-da4f-4b8e-b225-330b69323851 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.154774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 284/560] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:24.154774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ee7f1-da4f-4b8e-b225-330b69323851 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.203475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:24.217276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c77f9-59be-44f4-84bb-fc63590df9b6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.252685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc6278-13ae-4811-a3a1-6a6531255285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.253340 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc6278-13ae-4811-a3a1-6a6531255285 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.253610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ee7f1-da4f-4b8e-b225-330b69323851 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.253740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc6278-13ae-4811-a3a1-6a6531255285 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.253897 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bc6278-13ae-4811-a3a1-6a6531255285 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.254663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc6278-13ae-4811-a3a1-6a6531255285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:24.254970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc6278-13ae-4811-a3a1-6a6531255285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:24.256322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bc6278-13ae-4811-a3a1-6a6531255285 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:24.268594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 277/561] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:24.269219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc6278-13ae-4811-a3a1-6a6531255285 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.292424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f52948-5aa4-4a37-8b47-2178b38ccc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.293078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f52948-5aa4-4a37-8b47-2178b38ccc7c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.293361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f52948-5aa4-4a37-8b47-2178b38ccc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.293745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f52948-5aa4-4a37-8b47-2178b38ccc7c None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.294489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f52948-5aa4-4a37-8b47-2178b38ccc7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:24.294796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f52948-5aa4-4a37-8b47-2178b38ccc7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:24.296157 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f52948-5aa4-4a37-8b47-2178b38ccc7c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:24.312478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 285/562] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:24.312478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b97a9-42c8-4d0d-8491-0a1c6df0062f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.312478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b97a9-42c8-4d0d-8491-0a1c6df0062f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.312478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b97a9-42c8-4d0d-8491-0a1c6df0062f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.326382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-427b97a9-42c8-4d0d-8491-0a1c6df0062f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:24.327480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-427b97a9-42c8-4d0d-8491-0a1c6df0062f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:24.338290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:24.375924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bc6278-13ae-4811-a3a1-6a6531255285 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.406276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 286/563] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:24.406276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b97a9-42c8-4d0d-8491-0a1c6df0062f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.419229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5ec13-7945-47ec-8e43-9eb18d671dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.420204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5ec13-7945-47ec-8e43-9eb18d671dd4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.420512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5ec13-7945-47ec-8e43-9eb18d671dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.420838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e5ec13-7945-47ec-8e43-9eb18d671dd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.421581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5ec13-7945-47ec-8e43-9eb18d671dd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:24.421873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5ec13-7945-47ec-8e43-9eb18d671dd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:24.423406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e5ec13-7945-47ec-8e43-9eb18d671dd4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:24.432455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 278/564] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:24.433267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5ec13-7945-47ec-8e43-9eb18d671dd4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.457859 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:24.518434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b97a9-42c8-4d0d-8491-0a1c6df0062f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.539180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e5ec13-7945-47ec-8e43-9eb18d671dd4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.552842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479de6a-97a8-4acc-af33-f71c70332a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.553640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479de6a-97a8-4acc-af33-f71c70332a5b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.553927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479de6a-97a8-4acc-af33-f71c70332a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.554319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479de6a-97a8-4acc-af33-f71c70332a5b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.561430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479de6a-97a8-4acc-af33-f71c70332a5b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:24.562904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5479de6a-97a8-4acc-af33-f71c70332a5b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:24.566670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:24.575994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b076b9-9bab-4d68-89a6-65329b7d56b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.576836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b076b9-9bab-4d68-89a6-65329b7d56b5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.577067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b076b9-9bab-4d68-89a6-65329b7d56b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.577341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b076b9-9bab-4d68-89a6-65329b7d56b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5479de6a-97a8-4acc-af33-f71c70332a5b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:24.578211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:24.579095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:24.580005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 287/565] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:24.580581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479de6a-97a8-4acc-af33-f71c70332a5b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.590172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b076b9-9bab-4d68-89a6-65329b7d56b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:24.591606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b076b9-9bab-4d68-89a6-65329b7d56b5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-77b076b9-9bab-4d68-89a6-65329b7d56b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:24.606540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:24.607434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:24.608282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:24.608282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:24.608282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:24.608282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:24.608282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:24.608282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:24.608282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:24.608282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:24.608282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 279/566] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:24.608755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b076b9-9bab-4d68-89a6-65329b7d56b5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.687180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:24.690307 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479de6a-97a8-4acc-af33-f71c70332a5b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.715148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b076b9-9bab-4d68-89a6-65329b7d56b5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.726739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1821c8a-403d-496a-a5f9-acc70404d355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.727533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1821c8a-403d-496a-a5f9-acc70404d355 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.727815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1821c8a-403d-496a-a5f9-acc70404d355 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.728091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1821c8a-403d-496a-a5f9-acc70404d355 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.738350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1821c8a-403d-496a-a5f9-acc70404d355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b1821c8a-403d-496a-a5f9-acc70404d355 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:24.745957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:24.747087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:24.748148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:24.752113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 288/567] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:24.752926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1821c8a-403d-496a-a5f9-acc70404d355 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.753338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96ec26-cd2b-49f9-ac70-713284ae6b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.754316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96ec26-cd2b-49f9-ac70-713284ae6b71 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.754740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96ec26-cd2b-49f9-ac70-713284ae6b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.755154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f96ec26-cd2b-49f9-ac70-713284ae6b71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.766674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96ec26-cd2b-49f9-ac70-713284ae6b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:24.768353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f96ec26-cd2b-49f9-ac70-713284ae6b71 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9f96ec26-cd2b-49f9-ac70-713284ae6b71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:24.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:24.787969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:24.789050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:24.789050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:24.789050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:24.789050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:24.790622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 280/568] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:24.791576 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96ec26-cd2b-49f9-ac70-713284ae6b71 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.834450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:24.859703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1821c8a-403d-496a-a5f9-acc70404d355 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.897245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49541c42-4465-477c-a947-808793be9e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.898025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49541c42-4465-477c-a947-808793be9e7f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.898337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49541c42-4465-477c-a947-808793be9e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.898611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49541c42-4465-477c-a947-808793be9e7f None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.899439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49541c42-4465-477c-a947-808793be9e7f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:24.899752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49541c42-4465-477c-a947-808793be9e7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:24.901923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49541c42-4465-477c-a947-808793be9e7f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:24.904205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f96ec26-cd2b-49f9-ac70-713284ae6b71 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:24.925071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 289/569] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:24.925071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49541c42-4465-477c-a947-808793be9e7f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:24.944620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:24.945572 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ec3a6-58f5-4031-82e2-3c4b6b02f419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:24.946265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ec3a6-58f5-4031-82e2-3c4b6b02f419 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:24.946551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ec3a6-58f5-4031-82e2-3c4b6b02f419 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:24.946827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4ec3a6-58f5-4031-82e2-3c4b6b02f419 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:24.948995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ec3a6-58f5-4031-82e2-3c4b6b02f419 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:24.949259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ec3a6-58f5-4031-82e2-3c4b6b02f419 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:24.951085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4ec3a6-58f5-4031-82e2-3c4b6b02f419 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:24.964677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 281/570] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:24.965551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ec3a6-58f5-4031-82e2-3c4b6b02f419 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.027029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49541c42-4465-477c-a947-808793be9e7f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.059432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:25.060434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213567b-9c64-4e6b-b36f-9f785527eea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.061099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213567b-9c64-4e6b-b36f-9f785527eea2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.061424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213567b-9c64-4e6b-b36f-9f785527eea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.061665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213567b-9c64-4e6b-b36f-9f785527eea2 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.062443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213567b-9c64-4e6b-b36f-9f785527eea2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:25.062758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213567b-9c64-4e6b-b36f-9f785527eea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:25.062932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4ec3a6-58f5-4031-82e2-3c4b6b02f419 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.064111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b213567b-9c64-4e6b-b36f-9f785527eea2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:25.072320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 290/571] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:25.073713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213567b-9c64-4e6b-b36f-9f785527eea2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.097874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffbcec5-0d93-491d-b6d8-c2d05ed7d4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.098505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbcec5-0d93-491d-b6d8-c2d05ed7d4ef None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.098800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbcec5-0d93-491d-b6d8-c2d05ed7d4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.099073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffbcec5-0d93-491d-b6d8-c2d05ed7d4ef None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.099796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbcec5-0d93-491d-b6d8-c2d05ed7d4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:25.100105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbcec5-0d93-491d-b6d8-c2d05ed7d4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:25.101493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffbcec5-0d93-491d-b6d8-c2d05ed7d4ef None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:25.113161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 282/572] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:25.114546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c82e4-874e-4625-91f6-aab444c53058 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.114832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c82e4-874e-4625-91f6-aab444c53058 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.115106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c82e4-874e-4625-91f6-aab444c53058 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.118229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 283/573] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:25.118229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c82e4-874e-4625-91f6-aab444c53058 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.155191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:25.173795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213567b-9c64-4e6b-b36f-9f785527eea2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.205891 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011d22e-efa7-43fe-9a86-016c8a1c8d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.206697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011d22e-efa7-43fe-9a86-016c8a1c8d18 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.206981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011d22e-efa7-43fe-9a86-016c8a1c8d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.207267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b011d22e-efa7-43fe-9a86-016c8a1c8d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.217454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c82e4-874e-4625-91f6-aab444c53058 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.217956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011d22e-efa7-43fe-9a86-016c8a1c8d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:25.219432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b011d22e-efa7-43fe-9a86-016c8a1c8d18 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b011d22e-efa7-43fe-9a86-016c8a1c8d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:25.235792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:25.236673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:25.237518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:25.237518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:25.237518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:25.237518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:25.237518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 291/574] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:25.237792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011d22e-efa7-43fe-9a86-016c8a1c8d18 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.251078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b517fe8-3cde-4ce7-8887-678cb9c760ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.251734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b517fe8-3cde-4ce7-8887-678cb9c760ce None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.252009 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b517fe8-3cde-4ce7-8887-678cb9c760ce None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.252281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b517fe8-3cde-4ce7-8887-678cb9c760ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.253104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b517fe8-3cde-4ce7-8887-678cb9c760ce None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=60f006c99809427496b026c17a663995) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:25.253420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b517fe8-3cde-4ce7-8887-678cb9c760ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:25.254869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b517fe8-3cde-4ce7-8887-678cb9c760ce None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:25.260275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 284/575] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/role_assignments?user.id=60f006c99809427496b026c17a663995&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:25.261634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b517fe8-3cde-4ce7-8887-678cb9c760ce None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.290552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:25.362106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b011d22e-efa7-43fe-9a86-016c8a1c8d18 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.381945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b517fe8-3cde-4ce7-8887-678cb9c760ce None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.395766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:25.402394 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7efc4-7a40-4e55-90b4-3b4b21ada1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.403344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7efc4-7a40-4e55-90b4-3b4b21ada1ad None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.403455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7efc4-7a40-4e55-90b4-3b4b21ada1ad None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.403740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7efc4-7a40-4e55-90b4-3b4b21ada1ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.415629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7efc4-7a40-4e55-90b4-3b4b21ada1ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:25.417590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7efc4-7a40-4e55-90b4-3b4b21ada1ad None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:25.429966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0f4fe-e435-494b-9695-80dc51adfb62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.430876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f4fe-e435-494b-9695-80dc51adfb62 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.431160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f4fe-e435-494b-9695-80dc51adfb62 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.431437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d0f4fe-e435-494b-9695-80dc51adfb62 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-3ff7efc4-7a40-4e55-90b4-3b4b21ada1ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:25.435328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:25.436166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:25.441985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:25.441985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:25.441985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:25.441985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:25.441985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 292/576] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:25.441985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c97ea8-7b0a-4923-b141-94623a31cf7a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.441985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c97ea8-7b0a-4923-b141-94623a31cf7a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.441985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c97ea8-7b0a-4923-b141-94623a31cf7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.443522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0f4fe-e435-494b-9695-80dc51adfb62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:25.444992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d0f4fe-e435-494b-9695-80dc51adfb62 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:25.457827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-80c97ea8-7b0a-4923-b141-94623a31cf7a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:25.459024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-80c97ea8-7b0a-4923-b141-94623a31cf7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-06d0f4fe-e435-494b-9695-80dc51adfb62 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:25.460562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:25.461439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:25.462434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 285/577] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:25.463773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0f4fe-e435-494b-9695-80dc51adfb62 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.538366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 293/578] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:25.539729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c97ea8-7b0a-4923-b141-94623a31cf7a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.553893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:25.566460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d0f4fe-e435-494b-9695-80dc51adfb62 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.602188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b801b8-928d-4540-b3e9-ad39adc41672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.602998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b801b8-928d-4540-b3e9-ad39adc41672 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.603381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b801b8-928d-4540-b3e9-ad39adc41672 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.603754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b801b8-928d-4540-b3e9-ad39adc41672 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.605160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b801b8-928d-4540-b3e9-ad39adc41672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:25.605467 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b801b8-928d-4540-b3e9-ad39adc41672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:25.606845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b801b8-928d-4540-b3e9-ad39adc41672 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:25.618355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 286/579] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:25.623351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b801b8-928d-4540-b3e9-ad39adc41672 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.658937 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c97ea8-7b0a-4923-b141-94623a31cf7a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.674502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:25.697178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff49a2-a9c0-453a-a341-2b66653cd60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.697833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff49a2-a9c0-453a-a341-2b66653cd60a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.698484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff49a2-a9c0-453a-a341-2b66653cd60a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.698484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff49a2-a9c0-453a-a341-2b66653cd60a None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.699159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff49a2-a9c0-453a-a341-2b66653cd60a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:25.699468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff49a2-a9c0-453a-a341-2b66653cd60a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:25.700824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff49a2-a9c0-453a-a341-2b66653cd60a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:25.714314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 294/580] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:25.715096 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff49a2-a9c0-453a-a341-2b66653cd60a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.737169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b801b8-928d-4540-b3e9-ad39adc41672 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.771273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff1a29-a468-4706-af52-0b36b6efc514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.771965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff1a29-a468-4706-af52-0b36b6efc514 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.772251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff1a29-a468-4706-af52-0b36b6efc514 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.772523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff1a29-a468-4706-af52-0b36b6efc514 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.773358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff1a29-a468-4706-af52-0b36b6efc514 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:25.773668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff1a29-a468-4706-af52-0b36b6efc514 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:25.775121 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ff1a29-a468-4706-af52-0b36b6efc514 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:25.798418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 287/581] 10.222.0.13 () {56 vars in 1137 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:25.798418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff1a29-a468-4706-af52-0b36b6efc514 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.798418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:25.832069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff49a2-a9c0-453a-a341-2b66653cd60a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.873232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf099dc-be84-4c93-875c-04646f3e1f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.874244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf099dc-be84-4c93-875c-04646f3e1f9c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.874527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf099dc-be84-4c93-875c-04646f3e1f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.874793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf099dc-be84-4c93-875c-04646f3e1f9c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.886619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf099dc-be84-4c93-875c-04646f3e1f9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-bdf099dc-be84-4c93-875c-04646f3e1f9c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:25.894298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:25.895444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 295/582] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:25.900992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf099dc-be84-4c93-875c-04646f3e1f9c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:25.907887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ff1a29-a468-4706-af52-0b36b6efc514 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:25.935639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:25.949726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7de46-6d2b-4754-8a3f-e937779ae756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:25.950913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7de46-6d2b-4754-8a3f-e937779ae756 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:25.951266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7de46-6d2b-4754-8a3f-e937779ae756 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:25.951875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7de46-6d2b-4754-8a3f-e937779ae756 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:25.961210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7de46-6d2b-4754-8a3f-e937779ae756 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:25.963468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc7de46-6d2b-4754-8a3f-e937779ae756 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0bc7de46-6d2b-4754-8a3f-e937779ae756 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:25.982210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:25.983134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:25.983982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:25.983982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:25.983982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:25.983982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:25.983982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:25.983982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:25.983982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:25.983982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:25.983982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 288/583] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:25.984447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7de46-6d2b-4754-8a3f-e937779ae756 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.010917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf099dc-be84-4c93-875c-04646f3e1f9c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.046264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:26.047146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b420a0-45a3-4ef9-aad7-f84a8e5b8b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.047935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b420a0-45a3-4ef9-aad7-f84a8e5b8b7a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.048276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b420a0-45a3-4ef9-aad7-f84a8e5b8b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.048497 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b420a0-45a3-4ef9-aad7-f84a8e5b8b7a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.057681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b420a0-45a3-4ef9-aad7-f84a8e5b8b7a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.059171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b420a0-45a3-4ef9-aad7-f84a8e5b8b7a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-34b420a0-45a3-4ef9-aad7-f84a8e5b8b7a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:26.074022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:26.075170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 296/584] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:26.081296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b420a0-45a3-4ef9-aad7-f84a8e5b8b7a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.095390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7de46-6d2b-4754-8a3f-e937779ae756 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.130477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb1670-12d6-4c4d-8e65-d578c9fd4b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.131190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb1670-12d6-4c4d-8e65-d578c9fd4b0c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.131470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb1670-12d6-4c4d-8e65-d578c9fd4b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.131745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbb1670-12d6-4c4d-8e65-d578c9fd4b0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.143329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb1670-12d6-4c4d-8e65-d578c9fd4b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.144753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbb1670-12d6-4c4d-8e65-d578c9fd4b0c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-dfbb1670-12d6-4c4d-8e65-d578c9fd4b0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:26.163612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:26.164479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:26.165337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:26.165337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:26.165337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:26.165337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:26.165337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 289/585] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:26.165599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb1670-12d6-4c4d-8e65-d578c9fd4b0c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.180403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:26.188071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b420a0-45a3-4ef9-aad7-f84a8e5b8b7a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.223796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd9567-1163-4d44-b67f-47127acbe411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.224941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd9567-1163-4d44-b67f-47127acbe411 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.225309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd9567-1163-4d44-b67f-47127acbe411 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.225657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdd9567-1163-4d44-b67f-47127acbe411 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.226765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd9567-1163-4d44-b67f-47127acbe411 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:26.227145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd9567-1163-4d44-b67f-47127acbe411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.229299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdd9567-1163-4d44-b67f-47127acbe411 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:26.243780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 297/586] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:26.243780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd9567-1163-4d44-b67f-47127acbe411 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.272560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbb1670-12d6-4c4d-8e65-d578c9fd4b0c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.287509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:26.308787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9796d-4157-4326-bfd4-ecfd9dff6700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.309494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9796d-4157-4326-bfd4-ecfd9dff6700 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.309783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9796d-4157-4326-bfd4-ecfd9dff6700 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.310090 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f9796d-4157-4326-bfd4-ecfd9dff6700 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.310931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9796d-4157-4326-bfd4-ecfd9dff6700 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:26.311249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9796d-4157-4326-bfd4-ecfd9dff6700 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.312677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f9796d-4157-4326-bfd4-ecfd9dff6700 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:26.329901 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 290/587] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:26.329901 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9796d-4157-4326-bfd4-ecfd9dff6700 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.354960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdd9567-1163-4d44-b67f-47127acbe411 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.390472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049e2b4-4add-48b4-8558-669792200380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.391183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049e2b4-4add-48b4-8558-669792200380 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.391548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049e2b4-4add-48b4-8558-669792200380 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.391901 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049e2b4-4add-48b4-8558-669792200380 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.392846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049e2b4-4add-48b4-8558-669792200380 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:26.393235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049e2b4-4add-48b4-8558-669792200380 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.394797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049e2b4-4add-48b4-8558-669792200380 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:26.413673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 298/588] 10.222.0.13 () {56 vars in 1137 bytes} [Sun Jun 26 02:52:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:26.414598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049e2b4-4add-48b4-8558-669792200380 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.429492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:26.444165 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f9796d-4157-4326-bfd4-ecfd9dff6700 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.479426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8167ea6-3ebc-4fda-8f16-da0f37f9b966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.480085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8167ea6-3ebc-4fda-8f16-da0f37f9b966 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.480364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8167ea6-3ebc-4fda-8f16-da0f37f9b966 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.480633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8167ea6-3ebc-4fda-8f16-da0f37f9b966 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.481344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8167ea6-3ebc-4fda-8f16-da0f37f9b966 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:26.481648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8167ea6-3ebc-4fda-8f16-da0f37f9b966 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.483039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8167ea6-3ebc-4fda-8f16-da0f37f9b966 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:26.501855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 291/589] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:26.501855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8167ea6-3ebc-4fda-8f16-da0f37f9b966 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.522020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049e2b4-4add-48b4-8558-669792200380 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.543632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:26.558757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f167eb-b02a-4e20-9280-e7a5f699264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.559549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f167eb-b02a-4e20-9280-e7a5f699264e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.559831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f167eb-b02a-4e20-9280-e7a5f699264e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.560133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f167eb-b02a-4e20-9280-e7a5f699264e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.570435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f167eb-b02a-4e20-9280-e7a5f699264e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.571854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f167eb-b02a-4e20-9280-e7a5f699264e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-32f167eb-b02a-4e20-9280-e7a5f699264e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:26.589759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:26.591087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:26.592185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:26.592185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:26.592185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:26.592185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:26.592449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 299/590] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:26.593244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f167eb-b02a-4e20-9280-e7a5f699264e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.623981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8167ea6-3ebc-4fda-8f16-da0f37f9b966 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.659105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a0b15-f5d3-4005-9b3c-55a637010b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.659928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a0b15-f5d3-4005-9b3c-55a637010b20 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.660210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a0b15-f5d3-4005-9b3c-55a637010b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.660487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a0b15-f5d3-4005-9b3c-55a637010b20 None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/9ee0150dfb3f42709029e42d0fd9105d/roles/10b1c5eed2754da5adfd5217256adba8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.662419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:26.675337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a0b15-f5d3-4005-9b3c-55a637010b20 None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=9ee0150dfb3f42709029e42d0fd9105d, role_id=10b1c5eed2754da5adfd5217256adba8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.676776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233a0b15-f5d3-4005-9b3c-55a637010b20 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:26.697845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 292/591] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:52:26 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/9ee0150dfb3f42709029e42d0fd9105d/roles/10b1c5eed2754da5adfd5217256adba8 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:26.697845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a0b15-f5d3-4005-9b3c-55a637010b20 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.711825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f167eb-b02a-4e20-9280-e7a5f699264e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.757170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb459275-dadb-49d6-9da5-3128ffc8a305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.758428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb459275-dadb-49d6-9da5-3128ffc8a305 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.758876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb459275-dadb-49d6-9da5-3128ffc8a305 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.759251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb459275-dadb-49d6-9da5-3128ffc8a305 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.771881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb459275-dadb-49d6-9da5-3128ffc8a305 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.773919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb459275-dadb-49d6-9da5-3128ffc8a305 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-eb459275-dadb-49d6-9da5-3128ffc8a305 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:26.795454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:26.796556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:26.797626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:26.797626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:26.797626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:26.797626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:26.799393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 300/592] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:26.800423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb459275-dadb-49d6-9da5-3128ffc8a305 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.806158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:26.832183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a0b15-f5d3-4005-9b3c-55a637010b20 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.883729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b2054-db4c-4329-b857-66b8387599bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.884536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b2054-db4c-4329-b857-66b8387599bc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.884815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b2054-db4c-4329-b857-66b8387599bc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.885091 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949b2054-db4c-4329-b857-66b8387599bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.896023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b2054-db4c-4329-b857-66b8387599bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.897527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949b2054-db4c-4329-b857-66b8387599bc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-949b2054-db4c-4329-b857-66b8387599bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:26.912400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:26.913287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:26.914768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:26.914768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:26.914768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:26.914768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:26.914768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:26.914768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:26.914768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:26.914768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:26.914768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 293/593] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:26.914768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b2054-db4c-4329-b857-66b8387599bc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:26.935288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb459275-dadb-49d6-9da5-3128ffc8a305 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:26.958677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:26.980981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68827b1c-9be2-40ef-bf2a-6fa46f6cbf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:26.982204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68827b1c-9be2-40ef-bf2a-6fa46f6cbf6b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:26.982997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68827b1c-9be2-40ef-bf2a-6fa46f6cbf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:26.983767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68827b1c-9be2-40ef-bf2a-6fa46f6cbf6b None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:26.984860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68827b1c-9be2-40ef-bf2a-6fa46f6cbf6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:26.985688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68827b1c-9be2-40ef-bf2a-6fa46f6cbf6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:26.987773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68827b1c-9be2-40ef-bf2a-6fa46f6cbf6b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:27.005946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 301/594] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:27.007631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68827b1c-9be2-40ef-bf2a-6fa46f6cbf6b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:27.035026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949b2054-db4c-4329-b857-66b8387599bc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:27.075679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd6a0c-fcd7-45dd-97db-b33254232e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:27.076698 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd6a0c-fcd7-45dd-97db-b33254232e35 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.077704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd6a0c-fcd7-45dd-97db-b33254232e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.078228 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bd6a0c-fcd7-45dd-97db-b33254232e35 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.079418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd6a0c-fcd7-45dd-97db-b33254232e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:27.080178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd6a0c-fcd7-45dd-97db-b33254232e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:27.082516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bd6a0c-fcd7-45dd-97db-b33254232e35 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:27.101236 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 294/595] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:27.102996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd6a0c-fcd7-45dd-97db-b33254232e35 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:27.108051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:27.138176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68827b1c-9be2-40ef-bf2a-6fa46f6cbf6b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:27.188469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc04d47-728f-4be7-9687-263e81acbe9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:27.189153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc04d47-728f-4be7-9687-263e81acbe9a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.189608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc04d47-728f-4be7-9687-263e81acbe9a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.189958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc04d47-728f-4be7-9687-263e81acbe9a None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.190851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc04d47-728f-4be7-9687-263e81acbe9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:27.191232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc04d47-728f-4be7-9687-263e81acbe9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:27.192752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc04d47-728f-4be7-9687-263e81acbe9a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:27.214101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 302/596] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:27.214101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc04d47-728f-4be7-9687-263e81acbe9a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:27.225637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bd6a0c-fcd7-45dd-97db-b33254232e35 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:27.263604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b504350-fab2-467c-8d0e-1a18dea472a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:27.264719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b504350-fab2-467c-8d0e-1a18dea472a5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.265199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b504350-fab2-467c-8d0e-1a18dea472a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.265643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b504350-fab2-467c-8d0e-1a18dea472a5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.265752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:27.274752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b504350-fab2-467c-8d0e-1a18dea472a5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:27.276470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b504350-fab2-467c-8d0e-1a18dea472a5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9b504350-fab2-467c-8d0e-1a18dea472a5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:27.292097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:27.293206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:27.294368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:27.294368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:27.294368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:27.294368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:27.294368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:27.294368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:27.294368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:27.294368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:27.294368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 26 02:52:27.294368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:27.296823 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 295/597] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:27.297813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b504350-fab2-467c-8d0e-1a18dea472a5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:27.348276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc04d47-728f-4be7-9687-263e81acbe9a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:27.363191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:27.385025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3065a5-8468-40e6-ba21-1091eed32a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:27.386413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3065a5-8468-40e6-ba21-1091eed32a28 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.386694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3065a5-8468-40e6-ba21-1091eed32a28 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.386966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3065a5-8468-40e6-ba21-1091eed32a28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.393905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3065a5-8468-40e6-ba21-1091eed32a28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:27.395457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3065a5-8468-40e6-ba21-1091eed32a28 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:27.405105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b504350-fab2-467c-8d0e-1a18dea472a5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-6d3065a5-8468-40e6-ba21-1091eed32a28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:27.409439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:27.410570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:27.411640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:27.411640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:27.411640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:27.411640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:27.411640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:27.411640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:27.411640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:27.411640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:27.417761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 303/598] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:27.417761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3065a5-8468-40e6-ba21-1091eed32a28 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:27.451935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c550837-d5b6-459d-b182-a5b8f093b58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:27.452927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c550837-d5b6-459d-b182-a5b8f093b58f None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.453327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c550837-d5b6-459d-b182-a5b8f093b58f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.453672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c550837-d5b6-459d-b182-a5b8f093b58f None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/0dc5b601421a41d1a30a10e86967a920/roles/10b1c5eed2754da5adfd5217256adba8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.470481 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c550837-d5b6-459d-b182-a5b8f093b58f None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=0dc5b601421a41d1a30a10e86967a920, role_id=10b1c5eed2754da5adfd5217256adba8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:27.472003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c550837-d5b6-459d-b182-a5b8f093b58f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:27.486251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 296/599] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:52:27 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/0dc5b601421a41d1a30a10e86967a920/roles/10b1c5eed2754da5adfd5217256adba8 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:27.489380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61487e9-03a9-43a5-9fde-7d4c7d2af6f6 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.489916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61487e9-03a9-43a5-9fde-7d4c7d2af6f6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.490318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61487e9-03a9-43a5-9fde-7d4c7d2af6f6 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.494102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 297/600] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:27.494102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61487e9-03a9-43a5-9fde-7d4c7d2af6f6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:27.537280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:27.542814 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3065a5-8468-40e6-ba21-1091eed32a28 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:27.602106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2343aa3-a6d0-425e-a6a8-6e7088be811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:27.603430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2343aa3-a6d0-425e-a6a8-6e7088be811a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.605002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2343aa3-a6d0-425e-a6a8-6e7088be811a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.605840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2343aa3-a6d0-425e-a6a8-6e7088be811a None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.607208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2343aa3-a6d0-425e-a6a8-6e7088be811a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:27.608219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2343aa3-a6d0-425e-a6a8-6e7088be811a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:27.610002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2343aa3-a6d0-425e-a6a8-6e7088be811a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:27.626971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61487e9-03a9-43a5-9fde-7d4c7d2af6f6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:27.642082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 304/601] 10.222.0.13 () {56 vars in 1139 bytes} [Sun Jun 26 02:52:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:27.642082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a28f95-f697-48b6-94eb-62e9ddd9951d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.642082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a28f95-f697-48b6-94eb-62e9ddd9951d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.642082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a28f95-f697-48b6-94eb-62e9ddd9951d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.653717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:27.668326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f9a28f95-f697-48b6-94eb-62e9ddd9951d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:27.669895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f9a28f95-f697-48b6-94eb-62e9ddd9951d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:27.687940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4b976-6a4c-4a0d-a433-fae39cec9c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:27.688410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4b976-6a4c-4a0d-a433-fae39cec9c20 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.688832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4b976-6a4c-4a0d-a433-fae39cec9c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.689268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc4b976-6a4c-4a0d-a433-fae39cec9c20 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.690852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4b976-6a4c-4a0d-a433-fae39cec9c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:27.691190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4b976-6a4c-4a0d-a433-fae39cec9c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:27.692635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc4b976-6a4c-4a0d-a433-fae39cec9c20 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:27.706484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 298/602] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:27.706484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4b976-6a4c-4a0d-a433-fae39cec9c20 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:27.754489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 305/603] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:27.756045 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a28f95-f697-48b6-94eb-62e9ddd9951d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:27.817301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc4b976-6a4c-4a0d-a433-fae39cec9c20 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:27.860502 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8169f69-9c78-45a6-8ace-055c6ec65612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:27.861319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8169f69-9c78-45a6-8ace-055c6ec65612 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.861600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8169f69-9c78-45a6-8ace-055c6ec65612 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.861873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8169f69-9c78-45a6-8ace-055c6ec65612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.870638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:27.877160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8169f69-9c78-45a6-8ace-055c6ec65612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:27.879016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8169f69-9c78-45a6-8ace-055c6ec65612 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:27.891467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a28f95-f697-48b6-94eb-62e9ddd9951d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b8169f69-9c78-45a6-8ace-055c6ec65612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:27.901083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:27.902559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:27.903407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:27.903407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:27.903407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:27.903407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:27.903407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 299/604] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:27.904104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8169f69-9c78-45a6-8ace-055c6ec65612 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:27.934490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09278bec-cf10-423d-8fc7-46b2b1eba1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:27.935978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09278bec-cf10-423d-8fc7-46b2b1eba1be None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:27.936530 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09278bec-cf10-423d-8fc7-46b2b1eba1be None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:27.937002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09278bec-cf10-423d-8fc7-46b2b1eba1be None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/5e65f36f772b4220a909f8fda83ba42e/roles/10b1c5eed2754da5adfd5217256adba8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:27.940127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:27.957244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09278bec-cf10-423d-8fc7-46b2b1eba1be None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=5e65f36f772b4220a909f8fda83ba42e, role_id=10b1c5eed2754da5adfd5217256adba8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:27.959128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09278bec-cf10-423d-8fc7-46b2b1eba1be None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:27.978079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 306/605] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:52:27 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/5e65f36f772b4220a909f8fda83ba42e/roles/10b1c5eed2754da5adfd5217256adba8 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:27.980014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09278bec-cf10-423d-8fc7-46b2b1eba1be None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:28.020587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8169f69-9c78-45a6-8ace-055c6ec65612 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:28.056670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e365fc-e222-4607-b517-2186c32305f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:28.057533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e365fc-e222-4607-b517-2186c32305f5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:28.057812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e365fc-e222-4607-b517-2186c32305f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:28.058240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e365fc-e222-4607-b517-2186c32305f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:28.071740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e365fc-e222-4607-b517-2186c32305f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-97e365fc-e222-4607-b517-2186c32305f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:28.080022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:28.080998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:28.081845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 300/606] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:28.087858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e365fc-e222-4607-b517-2186c32305f5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:28.103232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09278bec-cf10-423d-8fc7-46b2b1eba1be None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:28.143908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ece38-cff5-4c90-b6e3-b4a63415768a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:28.145074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ece38-cff5-4c90-b6e3-b4a63415768a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:28.145588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ece38-cff5-4c90-b6e3-b4a63415768a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:28.146087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30ece38-cff5-4c90-b6e3-b4a63415768a None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:28.147031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ece38-cff5-4c90-b6e3-b4a63415768a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:28.147542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ece38-cff5-4c90-b6e3-b4a63415768a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:28.149162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30ece38-cff5-4c90-b6e3-b4a63415768a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:28.152747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:28.167731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 307/607] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:28.173041 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ece38-cff5-4c90-b6e3-b4a63415768a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:28.237535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:28.258384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e365fc-e222-4607-b517-2186c32305f5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:28.318132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3218e9-7fda-48c8-b3ff-a577cdb02f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:28.320890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3218e9-7fda-48c8-b3ff-a577cdb02f8f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:28.321386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3218e9-7fda-48c8-b3ff-a577cdb02f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:28.321881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3218e9-7fda-48c8-b3ff-a577cdb02f8f None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:28.322479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30ece38-cff5-4c90-b6e3-b4a63415768a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:28.323966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3218e9-7fda-48c8-b3ff-a577cdb02f8f None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:28.325847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3218e9-7fda-48c8-b3ff-a577cdb02f8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:28.327441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3218e9-7fda-48c8-b3ff-a577cdb02f8f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:28.344386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 301/608] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:28.351008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3218e9-7fda-48c8-b3ff-a577cdb02f8f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:28.374635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9caefc-4c13-4848-9a53-4538f6507574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:28.375489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9caefc-4c13-4848-9a53-4538f6507574 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:28.375776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9caefc-4c13-4848-9a53-4538f6507574 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:28.376053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9caefc-4c13-4848-9a53-4538f6507574 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:28.383533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9caefc-4c13-4848-9a53-4538f6507574 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:28.385049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9caefc-4c13-4848-9a53-4538f6507574 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-3f9caefc-4c13-4848-9a53-4538f6507574 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:28.400593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:28.402162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:28.403029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:28.403029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:28.403029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:28.403029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:28.403029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:28.403029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:28.403029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:28.403029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:28.407997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 308/609] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:28.409254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9caefc-4c13-4848-9a53-4538f6507574 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:28.471466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:28.508981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3218e9-7fda-48c8-b3ff-a577cdb02f8f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:28.518513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9caefc-4c13-4848-9a53-4538f6507574 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:28.544170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b524f-71b4-47b3-9900-9f1ed01ce422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:28.544963 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b524f-71b4-47b3-9900-9f1ed01ce422 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:28.545245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b524f-71b4-47b3-9900-9f1ed01ce422 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:28.545520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2b524f-71b4-47b3-9900-9f1ed01ce422 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:28.555633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aec004-e985-4739-8f49-e66da2332cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:28.556309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aec004-e985-4739-8f49-e66da2332cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:28.556417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b524f-71b4-47b3-9900-9f1ed01ce422 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:28.556597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aec004-e985-4739-8f49-e66da2332cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:28.556882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aec004-e985-4739-8f49-e66da2332cf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:28.557725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aec004-e985-4739-8f49-e66da2332cf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:28.557845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2b524f-71b4-47b3-9900-9f1ed01ce422 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:28.558056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aec004-e985-4739-8f49-e66da2332cf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:28.559697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aec004-e985-4739-8f49-e66da2332cf8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:28.566341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9f2b524f-71b4-47b3-9900-9f1ed01ce422 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:28.576559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:28.577472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:28.578382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:28.578382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:28.578382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:28.578382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:28.578382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 302/611] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:28.578657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 309/611] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:28.583097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b524f-71b4-47b3-9900-9f1ed01ce422 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:28.584867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aec004-e985-4739-8f49-e66da2332cf8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:28.686490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2b524f-71b4-47b3-9900-9f1ed01ce422 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:28.720611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f47335-5e4f-47bc-a3ec-50426a90cbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:28.721500 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f47335-5e4f-47bc-a3ec-50426a90cbb5 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:28.721785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f47335-5e4f-47bc-a3ec-50426a90cbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:28.722079 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f47335-5e4f-47bc-a3ec-50426a90cbb5 None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/8ff708d6758d4f7b913e763824f3b791/roles/10b1c5eed2754da5adfd5217256adba8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:28.736722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f47335-5e4f-47bc-a3ec-50426a90cbb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=8ff708d6758d4f7b913e763824f3b791, role_id=10b1c5eed2754da5adfd5217256adba8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:28.738198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f47335-5e4f-47bc-a3ec-50426a90cbb5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:28.756798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 303/612] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:52:28 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/8ff708d6758d4f7b913e763824f3b791/roles/10b1c5eed2754da5adfd5217256adba8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:28.825989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aec004-e985-4739-8f49-e66da2332cf8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:28.850653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:28.888978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b3b394-5099-45ac-aa50-61566c2d91e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:28.889721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b394-5099-45ac-aa50-61566c2d91e3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:28.890012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b394-5099-45ac-aa50-61566c2d91e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:28.890310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b3b394-5099-45ac-aa50-61566c2d91e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:28.891047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3b394-5099-45ac-aa50-61566c2d91e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:28.891358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3b394-5099-45ac-aa50-61566c2d91e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:28.892742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b3b394-5099-45ac-aa50-61566c2d91e3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:28.913505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 310/613] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:28.919457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f47335-5e4f-47bc-a3ec-50426a90cbb5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:28.929315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a328d-2a7b-46ad-a271-8e5430a43ee2 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:28.929606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a328d-2a7b-46ad-a271-8e5430a43ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:28.929887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6a328d-2a7b-46ad-a271-8e5430a43ee2 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:28.934158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 311/614] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:28 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:28.943511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b749f-2d1c-403c-b7a6-d39dd2e3b921 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:28.943810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b749f-2d1c-403c-b7a6-d39dd2e3b921 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:28.944081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b749f-2d1c-403c-b7a6-d39dd2e3b921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:28.966618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-537b749f-2d1c-403c-b7a6-d39dd2e3b921 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:28.967887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-537b749f-2d1c-403c-b7a6-d39dd2e3b921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:28.989214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:29.044524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 312/615] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:29.069846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f47335-5e4f-47bc-a3ec-50426a90cbb5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:29.075238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b749f-2d1c-403c-b7a6-d39dd2e3b921 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:29.119237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26edba-81f2-4d57-aaf3-16c0df6e00fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:29.120845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26edba-81f2-4d57-aaf3-16c0df6e00fd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:29.121273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26edba-81f2-4d57-aaf3-16c0df6e00fd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:29.121715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c26edba-81f2-4d57-aaf3-16c0df6e00fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:29.138350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26edba-81f2-4d57-aaf3-16c0df6e00fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:29.139816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c26edba-81f2-4d57-aaf3-16c0df6e00fd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2c26edba-81f2-4d57-aaf3-16c0df6e00fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:29.164478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:29.165640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:29.167334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 304/616] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:29.183322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26edba-81f2-4d57-aaf3-16c0df6e00fd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:29.240465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b749f-2d1c-403c-b7a6-d39dd2e3b921 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:29.251577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:29.310306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380575e-bc5d-4d5c-8f86-daa7ba6f7b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:29.311123 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380575e-bc5d-4d5c-8f86-daa7ba6f7b7e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:29.311411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380575e-bc5d-4d5c-8f86-daa7ba6f7b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:29.311690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f380575e-bc5d-4d5c-8f86-daa7ba6f7b7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:29.336315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f380575e-bc5d-4d5c-8f86-daa7ba6f7b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f380575e-bc5d-4d5c-8f86-daa7ba6f7b7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:29.344715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:29.345610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:29.346441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:29.351893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 313/617] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:29.358455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380575e-bc5d-4d5c-8f86-daa7ba6f7b7e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:29.365602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c26edba-81f2-4d57-aaf3-16c0df6e00fd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:29.384012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:29.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30542e-23e3-4deb-b33b-12a9cd3b5cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:29.430080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30542e-23e3-4deb-b33b-12a9cd3b5cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:29.430369 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30542e-23e3-4deb-b33b-12a9cd3b5cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:29.430648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30542e-23e3-4deb-b33b-12a9cd3b5cd8 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:29.431492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30542e-23e3-4deb-b33b-12a9cd3b5cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:29.431964 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30542e-23e3-4deb-b33b-12a9cd3b5cd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:29.433943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a30542e-23e3-4deb-b33b-12a9cd3b5cd8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:29.472609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 305/618] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Jun 26 02:52:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:29.479338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30542e-23e3-4deb-b33b-12a9cd3b5cd8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:29.511455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f380575e-bc5d-4d5c-8f86-daa7ba6f7b7e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:29.552006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c516364-cb84-4747-9a4c-6509e7dd4f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:29.552680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c516364-cb84-4747-9a4c-6509e7dd4f5f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:29.552964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c516364-cb84-4747-9a4c-6509e7dd4f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:29.553234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c516364-cb84-4747-9a4c-6509e7dd4f5f None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:29.553976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c516364-cb84-4747-9a4c-6509e7dd4f5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:29.554314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c516364-cb84-4747-9a4c-6509e7dd4f5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:29.555679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c516364-cb84-4747-9a4c-6509e7dd4f5f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:29.578148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 314/619] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:29.583394 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c516364-cb84-4747-9a4c-6509e7dd4f5f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:29.643039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a30542e-23e3-4deb-b33b-12a9cd3b5cd8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:29.644572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:29.689967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a303a8-d7fa-45b2-97a8-b51f21dab8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:29.690800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a303a8-d7fa-45b2-97a8-b51f21dab8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:29.691075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a303a8-d7fa-45b2-97a8-b51f21dab8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:29.691341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a303a8-d7fa-45b2-97a8-b51f21dab8c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:29.708365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a303a8-d7fa-45b2-97a8-b51f21dab8c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:29.710105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a303a8-d7fa-45b2-97a8-b51f21dab8c2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f6a303a8-d7fa-45b2-97a8-b51f21dab8c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:29.732004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:29.733109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:29.733970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:29.733970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:29.733970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:29.733970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:29.733970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 306/620] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:29.738897 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a303a8-d7fa-45b2-97a8-b51f21dab8c2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:29.780231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:29.803341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c516364-cb84-4747-9a4c-6509e7dd4f5f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:29.876950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b4dfa-9ca2-4a18-8a14-1bc9596e53a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:29.878612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4dfa-9ca2-4a18-8a14-1bc9596e53a2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:29.878894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4dfa-9ca2-4a18-8a14-1bc9596e53a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:29.879164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7b4dfa-9ca2-4a18-8a14-1bc9596e53a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:29.897872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b4dfa-9ca2-4a18-8a14-1bc9596e53a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:29.899396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7b4dfa-9ca2-4a18-8a14-1bc9596e53a2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0d7b4dfa-9ca2-4a18-8a14-1bc9596e53a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:29.917720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:29.918651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:29.919499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:29.919499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:29.919499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:29.919499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:29.919499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 315/621] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:29.924964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b4dfa-9ca2-4a18-8a14-1bc9596e53a2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:29.957914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a303a8-d7fa-45b2-97a8-b51f21dab8c2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:30.008773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9cae7-4fff-4b0a-86fd-67952ee76078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:30.009750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9cae7-4fff-4b0a-86fd-67952ee76078 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.010259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9cae7-4fff-4b0a-86fd-67952ee76078 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.010739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed9cae7-4fff-4b0a-86fd-67952ee76078 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.011683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9cae7-4fff-4b0a-86fd-67952ee76078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:30.012210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9cae7-4fff-4b0a-86fd-67952ee76078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:30.013762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed9cae7-4fff-4b0a-86fd-67952ee76078 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:30.026459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 307/622] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:30.034957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9cae7-4fff-4b0a-86fd-67952ee76078 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:30.096650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:30.122260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:30.140377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7b4dfa-9ca2-4a18-8a14-1bc9596e53a2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:30.151744 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed9cae7-4fff-4b0a-86fd-67952ee76078 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:30.188484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34daa765-4962-42f6-8310-b240343e0001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:30.189667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daa765-4962-42f6-8310-b240343e0001 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.190294 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daa765-4962-42f6-8310-b240343e0001 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.190847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daa765-4962-42f6-8310-b240343e0001 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.191621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34daa765-4962-42f6-8310-b240343e0001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:30.191974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34daa765-4962-42f6-8310-b240343e0001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:30.193325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34daa765-4962-42f6-8310-b240343e0001 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:30.195240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1911019-5460-4c79-a1a6-ea9bf4cc381b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:30.196331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1911019-5460-4c79-a1a6-ea9bf4cc381b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.196755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1911019-5460-4c79-a1a6-ea9bf4cc381b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.197170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1911019-5460-4c79-a1a6-ea9bf4cc381b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.204981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1911019-5460-4c79-a1a6-ea9bf4cc381b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:30.206647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1911019-5460-4c79-a1a6-ea9bf4cc381b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:30.208001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 316/623] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:30.209997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fd2e4-aa5c-4171-843e-d5385e664a9b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.210329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fd2e4-aa5c-4171-843e-d5385e664a9b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.210609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fd2e4-aa5c-4171-843e-d5385e664a9b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.216534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 317/624] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:30.217447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fd2e4-aa5c-4171-843e-d5385e664a9b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d1911019-5460-4c79-a1a6-ea9bf4cc381b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:30.225415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:30.226521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:30.227452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:30.227452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:30.227452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:30.227452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:30.227452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:30.227452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:30.227452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:30.227452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:30.234628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 308/625] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:30.236209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980c988-1092-4f20-b008-33cb30b25eb8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.236696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980c988-1092-4f20-b008-33cb30b25eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.237162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9980c988-1092-4f20-b008-33cb30b25eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.256520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9980c988-1092-4f20-b008-33cb30b25eb8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:30.257927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9980c988-1092-4f20-b008-33cb30b25eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:30.330228 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 309/626] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:30.331569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980c988-1092-4f20-b008-33cb30b25eb8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:30.350244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fd2e4-aa5c-4171-843e-d5385e664a9b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:30.404512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3fcd9-a98a-44f4-b3e4-92d6d6cfec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:30.405338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3fcd9-a98a-44f4-b3e4-92d6d6cfec5d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.405864 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3fcd9-a98a-44f4-b3e4-92d6d6cfec5d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.406167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d3fcd9-a98a-44f4-b3e4-92d6d6cfec5d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.415708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3fcd9-a98a-44f4-b3e4-92d6d6cfec5d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:30.418358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d3fcd9-a98a-44f4-b3e4-92d6d6cfec5d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:30.420130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:30.422630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-02d3fcd9-a98a-44f4-b3e4-92d6d6cfec5d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:30.435774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:30.436733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:30.437653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 318/627] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Jun 26 02:52:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:30.442788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3fcd9-a98a-44f4-b3e4-92d6d6cfec5d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:30.454475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9980c988-1092-4f20-b008-33cb30b25eb8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:30.489568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c077a9b-084f-49af-9ef1-23840c9bec65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:30.490263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c077a9b-084f-49af-9ef1-23840c9bec65 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.490539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c077a9b-084f-49af-9ef1-23840c9bec65 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.490813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c077a9b-084f-49af-9ef1-23840c9bec65 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.491592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c077a9b-084f-49af-9ef1-23840c9bec65 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:30.491898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c077a9b-084f-49af-9ef1-23840c9bec65 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:30.493311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c077a9b-084f-49af-9ef1-23840c9bec65 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:30.509994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 310/628] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:30.509994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c87d8-85b1-4c0c-9c6a-6d6c183e8c62 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.509994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c87d8-85b1-4c0c-9c6a-6d6c183e8c62 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.509994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883c87d8-85b1-4c0c-9c6a-6d6c183e8c62 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.509994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 311/629] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:30.509994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c87d8-85b1-4c0c-9c6a-6d6c183e8c62 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:30.552212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d3fcd9-a98a-44f4-b3e4-92d6d6cfec5d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:30.589854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58d4e8-7d90-4f73-8065-4e4c3771a920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:30.590883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58d4e8-7d90-4f73-8065-4e4c3771a920 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.591277 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58d4e8-7d90-4f73-8065-4e4c3771a920 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.591555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58d4e8-7d90-4f73-8065-4e4c3771a920 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.604050 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58d4e8-7d90-4f73-8065-4e4c3771a920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5f58d4e8-7d90-4f73-8065-4e4c3771a920 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:30.615095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:30.616009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:30.616847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:30.619278 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883c87d8-85b1-4c0c-9c6a-6d6c183e8c62 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:30.621576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 319/630] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:30.623200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58d4e8-7d90-4f73-8065-4e4c3771a920 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:30.651837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:30.658974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edf394e-7e3a-468b-a223-eaecb6cd6ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:30.659617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf394e-7e3a-468b-a223-eaecb6cd6ce9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.659904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf394e-7e3a-468b-a223-eaecb6cd6ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.660190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edf394e-7e3a-468b-a223-eaecb6cd6ce9 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.661012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf394e-7e3a-468b-a223-eaecb6cd6ce9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:30.661323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf394e-7e3a-468b-a223-eaecb6cd6ce9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:30.662770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edf394e-7e3a-468b-a223-eaecb6cd6ce9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:30.680880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 312/631] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Jun 26 02:52:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:30.687230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108503a-35de-43f5-8638-9482f1565d02 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.687526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108503a-35de-43f5-8638-9482f1565d02 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.687807 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d108503a-35de-43f5-8638-9482f1565d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.689175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:30.704395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d108503a-35de-43f5-8638-9482f1565d02 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:30.705589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d108503a-35de-43f5-8638-9482f1565d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:30.733162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58d4e8-7d90-4f73-8065-4e4c3771a920 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:30.770860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055e6be-3c66-4ee7-9c1b-3c515d8ce95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:30.772040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e6be-3c66-4ee7-9c1b-3c515d8ce95a None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.772389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e6be-3c66-4ee7-9c1b-3c515d8ce95a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.772611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055e6be-3c66-4ee7-9c1b-3c515d8ce95a None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/60f006c99809427496b026c17a663995/roles/10b1c5eed2754da5adfd5217256adba8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.774715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 313/632] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:30.775394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108503a-35de-43f5-8638-9482f1565d02 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:30.791545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055e6be-3c66-4ee7-9c1b-3c515d8ce95a None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=60f006c99809427496b026c17a663995, role_id=10b1c5eed2754da5adfd5217256adba8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:30.793583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2055e6be-3c66-4ee7-9c1b-3c515d8ce95a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:30.806591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 320/633] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:52:30 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/60f006c99809427496b026c17a663995/roles/10b1c5eed2754da5adfd5217256adba8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:30.808110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055e6be-3c66-4ee7-9c1b-3c515d8ce95a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:30.931069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2055e6be-3c66-4ee7-9c1b-3c515d8ce95a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:30.935126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d108503a-35de-43f5-8638-9482f1565d02 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:30.946872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:30.959913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:30.981345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dd9fd-8dde-4e14-aebc-610e6d1219fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:30.982107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dd9fd-8dde-4e14-aebc-610e6d1219fb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.982399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dd9fd-8dde-4e14-aebc-610e6d1219fb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.982750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2dd9fd-8dde-4e14-aebc-610e6d1219fb None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.983527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dd9fd-8dde-4e14-aebc-610e6d1219fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:30.983758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7784657-bbd3-4969-8715-e230c2b380e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:30.983873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dd9fd-8dde-4e14-aebc-610e6d1219fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:30.984718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7784657-bbd3-4969-8715-e230c2b380e5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:30.985194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7784657-bbd3-4969-8715-e230c2b380e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:30.985317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2dd9fd-8dde-4e14-aebc-610e6d1219fb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:30.985825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7784657-bbd3-4969-8715-e230c2b380e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:30.994601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 314/634] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:30.995798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dd9fd-8dde-4e14-aebc-610e6d1219fb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:31.062031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7784657-bbd3-4969-8715-e230c2b380e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:31.063971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7784657-bbd3-4969-8715-e230c2b380e5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:31.099839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2dd9fd-8dde-4e14-aebc-610e6d1219fb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:31.139957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4d265-4689-4480-9e31-595dde23dd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:31.140761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4d265-4689-4480-9e31-595dde23dd00 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.141047 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4d265-4689-4480-9e31-595dde23dd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.141341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4d265-4689-4480-9e31-595dde23dd00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.153221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4d265-4689-4480-9e31-595dde23dd00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8ca4d265-4689-4480-9e31-595dde23dd00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:31.161529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:31.162762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 315/635] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:31.163616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4d265-4689-4480-9e31-595dde23dd00 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:31.174062 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 321/636] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:31.175632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e250f-9284-495c-b3c0-314bc77f123d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.176004 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e250f-9284-495c-b3c0-314bc77f123d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.176362 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e250f-9284-495c-b3c0-314bc77f123d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.181565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 322/637] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:31.181565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e250f-9284-495c-b3c0-314bc77f123d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:31.194749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:31.218853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:31.278499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4d265-4689-4480-9e31-595dde23dd00 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:31.294143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e250f-9284-495c-b3c0-314bc77f123d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:31.313607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445a569-1982-427b-86bb-5c7b1cde4df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:31.314436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445a569-1982-427b-86bb-5c7b1cde4df6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.314723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445a569-1982-427b-86bb-5c7b1cde4df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.314999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b445a569-1982-427b-86bb-5c7b1cde4df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.326426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445a569-1982-427b-86bb-5c7b1cde4df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:31.327871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b445a569-1982-427b-86bb-5c7b1cde4df6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:31.333282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929dc574-3268-41bc-94e6-b5eb4603a459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:31.334712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dc574-3268-41bc-94e6-b5eb4603a459 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.335223 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dc574-3268-41bc-94e6-b5eb4603a459 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.335712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929dc574-3268-41bc-94e6-b5eb4603a459 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.337201 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929dc574-3268-41bc-94e6-b5eb4603a459 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:31.337804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929dc574-3268-41bc-94e6-b5eb4603a459 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:31.340079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929dc574-3268-41bc-94e6-b5eb4603a459 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b445a569-1982-427b-86bb-5c7b1cde4df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:31.345879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:31.346785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:31.347650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:31.347650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:31.347650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:31.347650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:31.347650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 316/638] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:31.347922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445a569-1982-427b-86bb-5c7b1cde4df6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:31.358593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 323/639] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:31.358593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ebd77-7112-4dd7-ae81-beec74641bfc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.358593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ebd77-7112-4dd7-ae81-beec74641bfc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.358593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ebd77-7112-4dd7-ae81-beec74641bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.373451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6e3ebd77-7112-4dd7-ae81-beec74641bfc None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:31.374923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6e3ebd77-7112-4dd7-ae81-beec74641bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:31.446274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:31.454102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 324/640] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:31.454102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ebd77-7112-4dd7-ae81-beec74641bfc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:31.455087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b445a569-1982-427b-86bb-5c7b1cde4df6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:31.462372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:31.491918 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc130-ed76-4d72-9cfd-132042ec12f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:31.492710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc130-ed76-4d72-9cfd-132042ec12f0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.492992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc130-ed76-4d72-9cfd-132042ec12f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44dc130-ed76-4d72-9cfd-132042ec12f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.504806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44dc130-ed76-4d72-9cfd-132042ec12f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:31.506290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44dc130-ed76-4d72-9cfd-132042ec12f0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b44dc130-ed76-4d72-9cfd-132042ec12f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:31.523266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:31.524164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:31.525030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:31.525030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:31.525030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:31.525030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:31.525030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 317/641] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:31.525298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc130-ed76-4d72-9cfd-132042ec12f0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:31.581621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ebd77-7112-4dd7-ae81-beec74641bfc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:31.616555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac325-23d2-43b8-9c6c-cf67c432fc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:31.617226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac325-23d2-43b8-9c6c-cf67c432fc4b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.617509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac325-23d2-43b8-9c6c-cf67c432fc4b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.617789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233ac325-23d2-43b8-9c6c-cf67c432fc4b None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.618561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ac325-23d2-43b8-9c6c-cf67c432fc4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:31.618880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ac325-23d2-43b8-9c6c-cf67c432fc4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:31.620886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233ac325-23d2-43b8-9c6c-cf67c432fc4b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:31.632986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 325/642] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:31.633769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac325-23d2-43b8-9c6c-cf67c432fc4b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:31.641725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44dc130-ed76-4d72-9cfd-132042ec12f0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:31.676186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f97e2c-e18e-43c8-8162-9ca1c12b0013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:31.676978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f97e2c-e18e-43c8-8162-9ca1c12b0013 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.677265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f97e2c-e18e-43c8-8162-9ca1c12b0013 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.677543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f97e2c-e18e-43c8-8162-9ca1c12b0013 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.688208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f97e2c-e18e-43c8-8162-9ca1c12b0013 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:31.689646 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f97e2c-e18e-43c8-8162-9ca1c12b0013 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:31.692177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:31.701196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-72f97e2c-e18e-43c8-8162-9ca1c12b0013 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:31.707662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:31.708586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:31.709448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:31.709448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:31.709448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:31.709448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:31.709448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 318/643] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:31.709735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f97e2c-e18e-43c8-8162-9ca1c12b0013 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:31.745570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233ac325-23d2-43b8-9c6c-cf67c432fc4b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:31.783145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae96af-a61e-4ffc-8ebe-8181fc821a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:31.784441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae96af-a61e-4ffc-8ebe-8181fc821a1f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.785716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae96af-a61e-4ffc-8ebe-8181fc821a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.786210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaae96af-a61e-4ffc-8ebe-8181fc821a1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.799343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaae96af-a61e-4ffc-8ebe-8181fc821a1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-aaae96af-a61e-4ffc-8ebe-8181fc821a1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:31.810388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:31.811269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:31.812126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:31.813513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 326/644] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:31.814261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae96af-a61e-4ffc-8ebe-8181fc821a1f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:31.822994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f97e2c-e18e-43c8-8162-9ca1c12b0013 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:31.871183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f117a-f14e-4d72-ae44-71e5f26dc6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:31.872189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f117a-f14e-4d72-ae44-71e5f26dc6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.872746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f117a-f14e-4d72-ae44-71e5f26dc6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.873094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3f117a-f14e-4d72-ae44-71e5f26dc6b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.874134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f117a-f14e-4d72-ae44-71e5f26dc6b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:31.874417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f117a-f14e-4d72-ae44-71e5f26dc6b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:31.875793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3f117a-f14e-4d72-ae44-71e5f26dc6b3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:31.888707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 319/645] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:31.889529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f117a-f14e-4d72-ae44-71e5f26dc6b3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:31.927112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaae96af-a61e-4ffc-8ebe-8181fc821a1f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:31.952597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:31.956557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:31.969000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46dd84-d479-4cb6-b2b7-7310fcceabad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:31.969873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46dd84-d479-4cb6-b2b7-7310fcceabad None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:31.971865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46dd84-d479-4cb6-b2b7-7310fcceabad None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:31.971865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46dd84-d479-4cb6-b2b7-7310fcceabad None admin] PATH_INFO: `/v3/users/glance` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:31.978375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46dd84-d479-4cb6-b2b7-7310fcceabad None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:31.980392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc46dd84-d479-4cb6-b2b7-7310fcceabad None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-cc46dd84-d479-4cb6-b2b7-7310fcceabad None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:31.994705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:31.995645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:31.996593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 327/646] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:31.997759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46dd84-d479-4cb6-b2b7-7310fcceabad None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.004427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3f117a-f14e-4d72-ae44-71e5f26dc6b3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:32.039962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcba13b-f43e-4410-835b-d1a28e9aa3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:32.040699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcba13b-f43e-4410-835b-d1a28e9aa3ed None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:32.040952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcba13b-f43e-4410-835b-d1a28e9aa3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:32.041268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcba13b-f43e-4410-835b-d1a28e9aa3ed None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:32.042624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcba13b-f43e-4410-835b-d1a28e9aa3ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:32.042979 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcba13b-f43e-4410-835b-d1a28e9aa3ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:32.044351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcba13b-f43e-4410-835b-d1a28e9aa3ed None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:32.060386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 320/647] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:32.060386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcba13b-f43e-4410-835b-d1a28e9aa3ed None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.106182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc46dd84-d479-4cb6-b2b7-7310fcceabad None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:32.141813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d69e7-3aca-4e3a-bcdc-1ae28d5bf61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:32.142657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d69e7-3aca-4e3a-bcdc-1ae28d5bf61f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:32.143003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d69e7-3aca-4e3a-bcdc-1ae28d5bf61f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:32.143341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385d69e7-3aca-4e3a-bcdc-1ae28d5bf61f None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:32.144142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d69e7-3aca-4e3a-bcdc-1ae28d5bf61f None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:32.144515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d69e7-3aca-4e3a-bcdc-1ae28d5bf61f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:32.146010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385d69e7-3aca-4e3a-bcdc-1ae28d5bf61f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:32.159615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 328/648] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:32.159615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d69e7-3aca-4e3a-bcdc-1ae28d5bf61f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.162130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcba13b-f43e-4410-835b-d1a28e9aa3ed None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:32.177818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:32.199910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b259c71-4913-468a-b617-8d823cfdf871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:32.200701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259c71-4913-468a-b617-8d823cfdf871 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:32.200990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259c71-4913-468a-b617-8d823cfdf871 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:32.201268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b259c71-4913-468a-b617-8d823cfdf871 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:32.207920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:32.209331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b259c71-4913-468a-b617-8d823cfdf871 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:32.211391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b259c71-4913-468a-b617-8d823cfdf871 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5b259c71-4913-468a-b617-8d823cfdf871 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:32.226864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:32.227961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:32.229361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:32.229361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:32.229361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:32.229361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:32.229361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:32.229361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:32.229361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:32.229361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:32.232710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 321/649] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:32.233419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b259c71-4913-468a-b617-8d823cfdf871 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.265942 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385d69e7-3aca-4e3a-bcdc-1ae28d5bf61f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:32.304704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c259eb-be79-4f66-8e13-8ecf66b74cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:32.305711 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c259eb-be79-4f66-8e13-8ecf66b74cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:32.305711 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c259eb-be79-4f66-8e13-8ecf66b74cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:32.305904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c259eb-be79-4f66-8e13-8ecf66b74cd8 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:32.306796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c259eb-be79-4f66-8e13-8ecf66b74cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:32.307094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c259eb-be79-4f66-8e13-8ecf66b74cd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:32.308553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c259eb-be79-4f66-8e13-8ecf66b74cd8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:32.326587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 329/650] 10.222.0.13 () {56 vars in 1137 bytes} [Sun Jun 26 02:52:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:32.327497 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c259eb-be79-4f66-8e13-8ecf66b74cd8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.341007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b259c71-4913-468a-b617-8d823cfdf871 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:32.379295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94200b-834a-4426-81da-dac683f1f20f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:32.380342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94200b-834a-4426-81da-dac683f1f20f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:32.380773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94200b-834a-4426-81da-dac683f1f20f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:32.381184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e94200b-834a-4426-81da-dac683f1f20f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:32.389062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94200b-834a-4426-81da-dac683f1f20f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:32.390720 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e94200b-834a-4426-81da-dac683f1f20f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5e94200b-834a-4426-81da-dac683f1f20f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:32.405862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:32.406835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:32.407742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 322/651] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:32 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:32.410010 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94200b-834a-4426-81da-dac683f1f20f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.437358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c259eb-be79-4f66-8e13-8ecf66b74cd8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:32.439226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:32.452048 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:32.478429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3936b-8fa2-40c1-a9e9-4a02c0961709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:32.479202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3936b-8fa2-40c1-a9e9-4a02c0961709 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:32.479993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3936b-8fa2-40c1-a9e9-4a02c0961709 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:32.480256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3936b-8fa2-40c1-a9e9-4a02c0961709 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:32.490275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3936b-8fa2-40c1-a9e9-4a02c0961709 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:32.491719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3936b-8fa2-40c1-a9e9-4a02c0961709 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-cef3936b-8fa2-40c1-a9e9-4a02c0961709 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:32.509687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:32.510844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:32.511821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:32.511821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:32.511821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:32.511821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:32.511821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 330/652] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:32.512087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3936b-8fa2-40c1-a9e9-4a02c0961709 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.531417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e94200b-834a-4426-81da-dac683f1f20f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:32.567592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48af13-311a-4f84-b091-9b1e8799e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:32.568374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48af13-311a-4f84-b091-9b1e8799e9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:32.568656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48af13-311a-4f84-b091-9b1e8799e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:32.568935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c48af13-311a-4f84-b091-9b1e8799e9f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:32.569891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48af13-311a-4f84-b091-9b1e8799e9f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:32.570259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48af13-311a-4f84-b091-9b1e8799e9f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:32.571829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c48af13-311a-4f84-b091-9b1e8799e9f2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:32.590772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 323/653] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:32.590772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48af13-311a-4f84-b091-9b1e8799e9f2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.617950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3936b-8fa2-40c1-a9e9-4a02c0961709 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:32.655113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeddc65-b462-454d-82d5-bf7dd3868497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:32.655895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeddc65-b462-454d-82d5-bf7dd3868497 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:32.656269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeddc65-b462-454d-82d5-bf7dd3868497 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:32.656599 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeddc65-b462-454d-82d5-bf7dd3868497 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:32.681868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:32.691624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:32.698943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c48af13-311a-4f84-b091-9b1e8799e9f2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:32.735801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea2149-f0f9-4fa8-8e2a-1311ac7e47e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:32.736498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea2149-f0f9-4fa8-8e2a-1311ac7e47e5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:32.736856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea2149-f0f9-4fa8-8e2a-1311ac7e47e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:32.737222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ea2149-f0f9-4fa8-8e2a-1311ac7e47e5 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:32.738167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea2149-f0f9-4fa8-8e2a-1311ac7e47e5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:32.738546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea2149-f0f9-4fa8-8e2a-1311ac7e47e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:32.740056 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ea2149-f0f9-4fa8-8e2a-1311ac7e47e5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:32.742853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeddc65-b462-454d-82d5-bf7dd3868497 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:32.745050 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeddc65-b462-454d-82d5-bf7dd3868497 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:32.758360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 324/654] 10.222.0.13 () {56 vars in 1137 bytes} [Sun Jun 26 02:52:32 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:32.763863 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea2149-f0f9-4fa8-8e2a-1311ac7e47e5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.862906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 331/655] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:32.863768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeddc65-b462-454d-82d5-bf7dd3868497 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.875871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ea2149-f0f9-4fa8-8e2a-1311ac7e47e5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:32.910515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece3fe0-ab80-49ae-8897-4d7ee8f3d300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:32.911169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece3fe0-ab80-49ae-8897-4d7ee8f3d300 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:32.911451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece3fe0-ab80-49ae-8897-4d7ee8f3d300 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:32.911717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ece3fe0-ab80-49ae-8897-4d7ee8f3d300 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:32.912424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece3fe0-ab80-49ae-8897-4d7ee8f3d300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:32.912727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece3fe0-ab80-49ae-8897-4d7ee8f3d300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:32.914104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ece3fe0-ab80-49ae-8897-4d7ee8f3d300 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:32.917976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:32.925238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 325/656] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:32.926024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece3fe0-ab80-49ae-8897-4d7ee8f3d300 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:32.941288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:32.966698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeddc65-b462-454d-82d5-bf7dd3868497 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:33.000391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c4132-c281-463d-b880-cb477f0d8f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:33.000975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c4132-c281-463d-b880-cb477f0d8f40 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.001258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c4132-c281-463d-b880-cb477f0d8f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.001867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c4132-c281-463d-b880-cb477f0d8f40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.002728 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c4132-c281-463d-b880-cb477f0d8f40 None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=9ee0150dfb3f42709029e42d0fd9105d) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:33.003348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c4132-c281-463d-b880-cb477f0d8f40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:33.004814 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c4132-c281-463d-b880-cb477f0d8f40 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:33.011017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 332/657] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:32 2022] GET /identity/v3/role_assignments?user.id=9ee0150dfb3f42709029e42d0fd9105d&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:33.012202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fcdfb-a1a1-4f03-b479-1885b94a595b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.012483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fcdfb-a1a1-4f03-b479-1885b94a595b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.012756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fcdfb-a1a1-4f03-b479-1885b94a595b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.014000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 333/658] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:33.014723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fcdfb-a1a1-4f03-b479-1885b94a595b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:33.034977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ece3fe0-ab80-49ae-8897-4d7ee8f3d300 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:33.080785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfadcda-8b18-4c90-8aa7-5cd3745e29c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:33.081463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfadcda-8b18-4c90-8aa7-5cd3745e29c4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.081745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfadcda-8b18-4c90-8aa7-5cd3745e29c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.082029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfadcda-8b18-4c90-8aa7-5cd3745e29c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.138636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fcdfb-a1a1-4f03-b479-1885b94a595b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:33.158321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:33.161435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfadcda-8b18-4c90-8aa7-5cd3745e29c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:33.162646 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfadcda-8b18-4c90-8aa7-5cd3745e29c4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:33.179522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042dfa9-2899-4516-8672-8cb150ee73b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:33.179874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:33.180662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042dfa9-2899-4516-8672-8cb150ee73b7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.181066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042dfa9-2899-4516-8672-8cb150ee73b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.181218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8042dfa9-2899-4516-8672-8cb150ee73b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.192363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042dfa9-2899-4516-8672-8cb150ee73b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:33.193792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8042dfa9-2899-4516-8672-8cb150ee73b7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-8042dfa9-2899-4516-8672-8cb150ee73b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:33.212207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:33.213264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:33.214288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:33.214288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:33.214288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:33.214288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:33.218101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 334/659] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:33.218915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042dfa9-2899-4516-8672-8cb150ee73b7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:33.289212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 326/660] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:33.289212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fcbb4-c90c-4577-b7b8-3e78e092e4f1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.289212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fcbb4-c90c-4577-b7b8-3e78e092e4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.289212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39fcbb4-c90c-4577-b7b8-3e78e092e4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.303553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e39fcbb4-c90c-4577-b7b8-3e78e092e4f1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:33.304562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e39fcbb4-c90c-4577-b7b8-3e78e092e4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:33.332376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8042dfa9-2899-4516-8672-8cb150ee73b7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:33.367409 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f52f-1ba1-4da8-b106-5a8f394ef37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:33.368308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f52f-1ba1-4da8-b106-5a8f394ef37a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.368664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f52f-1ba1-4da8-b106-5a8f394ef37a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.368944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5122f52f-1ba1-4da8-b106-5a8f394ef37a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.377375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 327/661] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:33.377729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122f52f-1ba1-4da8-b106-5a8f394ef37a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:33.378757 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fcbb4-c90c-4577-b7b8-3e78e092e4f1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:33.379206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5122f52f-1ba1-4da8-b106-5a8f394ef37a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5122f52f-1ba1-4da8-b106-5a8f394ef37a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:33.404810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:33.405789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:33.406778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 335/662] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:33.407374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f52f-1ba1-4da8-b106-5a8f394ef37a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:33.430688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:33.440524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:33.487859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39fcbb4-c90c-4577-b7b8-3e78e092e4f1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:33.517101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5122f52f-1ba1-4da8-b106-5a8f394ef37a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:33.524644 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c0094-85a4-47d5-b139-8bf2bf78db77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:33.525308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c0094-85a4-47d5-b139-8bf2bf78db77 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.525584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c0094-85a4-47d5-b139-8bf2bf78db77 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.525861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c0094-85a4-47d5-b139-8bf2bf78db77 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.526674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c0094-85a4-47d5-b139-8bf2bf78db77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:33.526980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c0094-85a4-47d5-b139-8bf2bf78db77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:33.528332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c0094-85a4-47d5-b139-8bf2bf78db77 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:33.539713 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 328/663] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:33.540453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c0094-85a4-47d5-b139-8bf2bf78db77 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:33.560933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51ac0d-2481-4ffc-854e-3cf2bcf23c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:33.561779 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51ac0d-2481-4ffc-854e-3cf2bcf23c5f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.562085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51ac0d-2481-4ffc-854e-3cf2bcf23c5f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.562385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d51ac0d-2481-4ffc-854e-3cf2bcf23c5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.573364 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51ac0d-2481-4ffc-854e-3cf2bcf23c5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:33.574999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d51ac0d-2481-4ffc-854e-3cf2bcf23c5f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-8d51ac0d-2481-4ffc-854e-3cf2bcf23c5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:33.592602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:33.593488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:33.594404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:33.594404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:33.594404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:33.594404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:33.594404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 336/664] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:33.594669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51ac0d-2481-4ffc-854e-3cf2bcf23c5f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:33.660293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c0094-85a4-47d5-b139-8bf2bf78db77 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:33.663760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:33.682975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:33.696907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1d3f0-1687-49b0-a61f-785b86349746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:33.697761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1d3f0-1687-49b0-a61f-785b86349746 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.698106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1d3f0-1687-49b0-a61f-785b86349746 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.698389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1d3f0-1687-49b0-a61f-785b86349746 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.703168 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d51ac0d-2481-4ffc-854e-3cf2bcf23c5f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:33.709701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1d3f0-1687-49b0-a61f-785b86349746 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ebf1d3f0-1687-49b0-a61f-785b86349746 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:33.717501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:33.718448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:33.719255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 329/665] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:33.719927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1d3f0-1687-49b0-a61f-785b86349746 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:33.744687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0ca67-4b93-48f3-8ccf-2702e30443fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:33.745889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0ca67-4b93-48f3-8ccf-2702e30443fa None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.746197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0ca67-4b93-48f3-8ccf-2702e30443fa None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.746468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0ca67-4b93-48f3-8ccf-2702e30443fa None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.747304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0ca67-4b93-48f3-8ccf-2702e30443fa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:33.747606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0ca67-4b93-48f3-8ccf-2702e30443fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:33.749289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0ca67-4b93-48f3-8ccf-2702e30443fa None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:33.771323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 337/666] 10.222.0.13 () {56 vars in 1139 bytes} [Sun Jun 26 02:52:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:33.771323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0ca67-4b93-48f3-8ccf-2702e30443fa None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:33.835869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1d3f0-1687-49b0-a61f-785b86349746 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:33.876390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0ca67-4b93-48f3-8ccf-2702e30443fa None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:33.876859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe559bb4-3014-4f0e-8f57-948a44f7aa8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:33.877661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe559bb4-3014-4f0e-8f57-948a44f7aa8c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.877939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe559bb4-3014-4f0e-8f57-948a44f7aa8c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.878259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe559bb4-3014-4f0e-8f57-948a44f7aa8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.885158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe559bb4-3014-4f0e-8f57-948a44f7aa8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:33.886641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe559bb4-3014-4f0e-8f57-948a44f7aa8c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-fe559bb4-3014-4f0e-8f57-948a44f7aa8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:33.900647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:33.902030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:33.902942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:33.902942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:33.902942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:33.902942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:33.902942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:33.902942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:33.902942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:33.902942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:33.907002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 330/667] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:33.907820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe559bb4-3014-4f0e-8f57-948a44f7aa8c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:33.914010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7a1a5-84a4-4f4f-82ec-635864788940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:33.914706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7a1a5-84a4-4f4f-82ec-635864788940 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:33.914987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7a1a5-84a4-4f4f-82ec-635864788940 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:33.915257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce7a1a5-84a4-4f4f-82ec-635864788940 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:33.916130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7a1a5-84a4-4f4f-82ec-635864788940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:33.916304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7a1a5-84a4-4f4f-82ec-635864788940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:33.918187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce7a1a5-84a4-4f4f-82ec-635864788940 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:33.927308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:33.935297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:33.935959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 338/668] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:33.936772 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7a1a5-84a4-4f4f-82ec-635864788940 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:34.013829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe559bb4-3014-4f0e-8f57-948a44f7aa8c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:34.049608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ffc64-0194-40ae-8fe6-5ca9687e46a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:34.050310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce7a1a5-84a4-4f4f-82ec-635864788940 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:34.050649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ffc64-0194-40ae-8fe6-5ca9687e46a7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:34.050649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ffc64-0194-40ae-8fe6-5ca9687e46a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:34.050968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03ffc64-0194-40ae-8fe6-5ca9687e46a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:34.051592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ffc64-0194-40ae-8fe6-5ca9687e46a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:34.051898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ffc64-0194-40ae-8fe6-5ca9687e46a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:34.053265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03ffc64-0194-40ae-8fe6-5ca9687e46a7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:34.066450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 331/669] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:34.066450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ffc64-0194-40ae-8fe6-5ca9687e46a7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:34.092036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d2f88-a7c7-47f2-8832-bd9d202a13f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:34.092622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2f88-a7c7-47f2-8832-bd9d202a13f7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:34.092908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2f88-a7c7-47f2-8832-bd9d202a13f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:34.093188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532d2f88-a7c7-47f2-8832-bd9d202a13f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:34.154742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:34.163266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:34.165860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d2f88-a7c7-47f2-8832-bd9d202a13f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:34.168418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532d2f88-a7c7-47f2-8832-bd9d202a13f7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:34.173599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03ffc64-0194-40ae-8fe6-5ca9687e46a7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:34.211199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244868e-7bc9-4725-802b-c980845e2a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:34.211982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244868e-7bc9-4725-802b-c980845e2a01 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:34.212271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244868e-7bc9-4725-802b-c980845e2a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:34.212553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7244868e-7bc9-4725-802b-c980845e2a01 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:34.213347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244868e-7bc9-4725-802b-c980845e2a01 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:34.214226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244868e-7bc9-4725-802b-c980845e2a01 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:34.215899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7244868e-7bc9-4725-802b-c980845e2a01 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:34.234206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 332/670] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:34.234206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244868e-7bc9-4725-802b-c980845e2a01 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:34.278218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 339/671] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:34.279295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d2f88-a7c7-47f2-8832-bd9d202a13f7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:34.344433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7244868e-7bc9-4725-802b-c980845e2a01 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:34.381555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89483214-1fb9-499a-8634-b572d7cf1de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:34.382299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:34.382624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89483214-1fb9-499a-8634-b572d7cf1de9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:34.383005 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89483214-1fb9-499a-8634-b572d7cf1de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:34.383414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89483214-1fb9-499a-8634-b572d7cf1de9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:34.387535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532d2f88-a7c7-47f2-8832-bd9d202a13f7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:34.391099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89483214-1fb9-499a-8634-b572d7cf1de9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:34.392552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89483214-1fb9-499a-8634-b572d7cf1de9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-89483214-1fb9-499a-8634-b572d7cf1de9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:34.406708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:34.407729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:34.414071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:34.414071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:34.414071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:34.414071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:34.414071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:34.414071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:34.414071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:34.414071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:34.414071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 333/672] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:34.414071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89483214-1fb9-499a-8634-b572d7cf1de9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:34.420958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:34.429826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c5950-5f1a-4a7b-866f-2a0dece320fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:34.430736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c5950-5f1a-4a7b-866f-2a0dece320fe None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:34.431089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c5950-5f1a-4a7b-866f-2a0dece320fe None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:34.431434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75c5950-5f1a-4a7b-866f-2a0dece320fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:34.441645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c5950-5f1a-4a7b-866f-2a0dece320fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:34.443153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75c5950-5f1a-4a7b-866f-2a0dece320fe None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b75c5950-5f1a-4a7b-866f-2a0dece320fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:34.461762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:34.462815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:34.464272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:34.464272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:34.464272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:34.464272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:34.464272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 340/673] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:34.464566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c5950-5f1a-4a7b-866f-2a0dece320fe None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:34.528293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89483214-1fb9-499a-8634-b572d7cf1de9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:34.566651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534b959-b124-4e41-a26e-f2f5c0b5bc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:34.567309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534b959-b124-4e41-a26e-f2f5c0b5bc4e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:34.567632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534b959-b124-4e41-a26e-f2f5c0b5bc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:34.567903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e534b959-b124-4e41-a26e-f2f5c0b5bc4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:34.568683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534b959-b124-4e41-a26e-f2f5c0b5bc4e None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=0dc5b601421a41d1a30a10e86967a920) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:34.569006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534b959-b124-4e41-a26e-f2f5c0b5bc4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:34.570527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e534b959-b124-4e41-a26e-f2f5c0b5bc4e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:34.572573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75c5950-5f1a-4a7b-866f-2a0dece320fe None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:34.586084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 334/674] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:34 2022] GET /identity/v3/role_assignments?user.id=0dc5b601421a41d1a30a10e86967a920&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:34.586084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534b959-b124-4e41-a26e-f2f5c0b5bc4e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:34.610534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ce9ce-7ba7-483b-91c6-87d2d0c40db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:34.611262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ce9ce-7ba7-483b-91c6-87d2d0c40db6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:34.611547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ce9ce-7ba7-483b-91c6-87d2d0c40db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:34.611822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71ce9ce-7ba7-483b-91c6-87d2d0c40db6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:34.624569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ce9ce-7ba7-483b-91c6-87d2d0c40db6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:34.626059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71ce9ce-7ba7-483b-91c6-87d2d0c40db6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-d71ce9ce-7ba7-483b-91c6-87d2d0c40db6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:34.655251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:34.656297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:34.657250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:34.657250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:34.657250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:34.657250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:34.658217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 341/675] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:34.659335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ce9ce-7ba7-483b-91c6-87d2d0c40db6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:34.664241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:34.689179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:34.725622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e534b959-b124-4e41-a26e-f2f5c0b5bc4e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:34.773342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b22f11-6bcb-49c2-8330-b50965a31098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:34.774022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b22f11-6bcb-49c2-8330-b50965a31098 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:34.774334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b22f11-6bcb-49c2-8330-b50965a31098 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:34.774622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b22f11-6bcb-49c2-8330-b50965a31098 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:34.775407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b22f11-6bcb-49c2-8330-b50965a31098 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:34.775718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b22f11-6bcb-49c2-8330-b50965a31098 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:34.777321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b22f11-6bcb-49c2-8330-b50965a31098 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:34.792503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 335/676] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:34.793549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b22f11-6bcb-49c2-8330-b50965a31098 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:34.802317 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71ce9ce-7ba7-483b-91c6-87d2d0c40db6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:34.844608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fd98c-6b82-4263-9e7e-0ace664c050f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:34.845782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fd98c-6b82-4263-9e7e-0ace664c050f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:34.846097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fd98c-6b82-4263-9e7e-0ace664c050f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:34.846377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fd98c-6b82-4263-9e7e-0ace664c050f None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:34.850222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7fd98c-6b82-4263-9e7e-0ace664c050f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:34.850222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7fd98c-6b82-4263-9e7e-0ace664c050f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:34.850222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7fd98c-6b82-4263-9e7e-0ace664c050f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:34.868633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 342/677] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:34.868633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fd98c-6b82-4263-9e7e-0ace664c050f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:34.927355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b22f11-6bcb-49c2-8330-b50965a31098 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:34.952068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:34.961862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:34.970174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd18d8b-5d6f-4704-989d-13d08f091da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:34.971117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd18d8b-5d6f-4704-989d-13d08f091da6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:34.971561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd18d8b-5d6f-4704-989d-13d08f091da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:34.972000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd18d8b-5d6f-4704-989d-13d08f091da6 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:34.972959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd18d8b-5d6f-4704-989d-13d08f091da6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:34.973427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd18d8b-5d6f-4704-989d-13d08f091da6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:34.975096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd18d8b-5d6f-4704-989d-13d08f091da6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:34.988692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 336/678] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:34.989833 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd18d8b-5d6f-4704-989d-13d08f091da6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:35.009964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fd98c-6b82-4263-9e7e-0ace664c050f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:35.053802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b470e6-6c67-40c8-bdc9-92f5c9bd36bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:35.054512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b470e6-6c67-40c8-bdc9-92f5c9bd36bc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:35.054891 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b470e6-6c67-40c8-bdc9-92f5c9bd36bc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:35.055245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b470e6-6c67-40c8-bdc9-92f5c9bd36bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:35.056298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b470e6-6c67-40c8-bdc9-92f5c9bd36bc None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=5e65f36f772b4220a909f8fda83ba42e) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:35.056689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b470e6-6c67-40c8-bdc9-92f5c9bd36bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:35.059346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b470e6-6c67-40c8-bdc9-92f5c9bd36bc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:35.074071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 343/679] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:34 2022] GET /identity/v3/role_assignments?user.id=5e65f36f772b4220a909f8fda83ba42e&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:35.074071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b470e6-6c67-40c8-bdc9-92f5c9bd36bc None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:35.144067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd18d8b-5d6f-4704-989d-13d08f091da6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:35.184532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a086bb-c3c9-410e-855d-3dd2b20105dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:35.185349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a086bb-c3c9-410e-855d-3dd2b20105dc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:35.185630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a086bb-c3c9-410e-855d-3dd2b20105dc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:35.185906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a086bb-c3c9-410e-855d-3dd2b20105dc None admin] PATH_INFO: `/v3/users/swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:35.194365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a086bb-c3c9-410e-855d-3dd2b20105dc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:35.195860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a086bb-c3c9-410e-855d-3dd2b20105dc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c3a086bb-c3c9-410e-855d-3dd2b20105dc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:35.211374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:35.212394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:35.213332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 337/680] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:35.214582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e4ac3-a1e1-4636-b455-c9e52827149f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:35.214763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e4ac3-a1e1-4636-b455-c9e52827149f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:35.216218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e4ac3-a1e1-4636-b455-c9e52827149f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:35.220533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 338/681] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:35.221455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e4ac3-a1e1-4636-b455-c9e52827149f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:35.231246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b470e6-6c67-40c8-bdc9-92f5c9bd36bc None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:35.261872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:35.271520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09a9eb-206d-4af8-8fbf-7b98e93b4c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:35.273758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09a9eb-206d-4af8-8fbf-7b98e93b4c62 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:35.273758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09a9eb-206d-4af8-8fbf-7b98e93b4c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:35.273758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09a9eb-206d-4af8-8fbf-7b98e93b4c62 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:35.282077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09a9eb-206d-4af8-8fbf-7b98e93b4c62 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:35.283793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09a9eb-206d-4af8-8fbf-7b98e93b4c62 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:35.284505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-2e09a9eb-206d-4af8-8fbf-7b98e93b4c62 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:35.299580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:35.300560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:35.301447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:35.301447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:35.301447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:35.301447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:35.301447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:35.301447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:35.301447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:35.301447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:35.301447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 344/682] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:35.302557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6a98c-2255-4a15-a19e-1c4f07b440ff None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:35.302557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6a98c-2255-4a15-a19e-1c4f07b440ff None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:35.302744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b6a98c-2255-4a15-a19e-1c4f07b440ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:35.322744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d7b6a98c-2255-4a15-a19e-1c4f07b440ff None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:35.324301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d7b6a98c-2255-4a15-a19e-1c4f07b440ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:35.361435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e4ac3-a1e1-4636-b455-c9e52827149f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:35.404705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819764a-dcf4-4bdb-8cbc-2f4c37ed1204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:35.405301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819764a-dcf4-4bdb-8cbc-2f4c37ed1204 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:35.405581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819764a-dcf4-4bdb-8cbc-2f4c37ed1204 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:35.405786 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 345/683] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:35.406763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6a98c-2255-4a15-a19e-1c4f07b440ff None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:35.410338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f819764a-dcf4-4bdb-8cbc-2f4c37ed1204 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:35.411232 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819764a-dcf4-4bdb-8cbc-2f4c37ed1204 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:35.411552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819764a-dcf4-4bdb-8cbc-2f4c37ed1204 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:35.413001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f819764a-dcf4-4bdb-8cbc-2f4c37ed1204 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:35.444122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 339/684] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Jun 26 02:52:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:35.445350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819764a-dcf4-4bdb-8cbc-2f4c37ed1204 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:35.540049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b6a98c-2255-4a15-a19e-1c4f07b440ff None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:35.562493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:35.571207 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f819764a-dcf4-4bdb-8cbc-2f4c37ed1204 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:35.581915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165a5d0-0a11-44b0-a5eb-2cfc92649c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:35.582665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a5d0-0a11-44b0-a5eb-2cfc92649c45 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:35.583230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:35.583467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a5d0-0a11-44b0-a5eb-2cfc92649c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:35.584222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1165a5d0-0a11-44b0-a5eb-2cfc92649c45 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:35.585251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165a5d0-0a11-44b0-a5eb-2cfc92649c45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:35.585710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165a5d0-0a11-44b0-a5eb-2cfc92649c45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:35.587359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1165a5d0-0a11-44b0-a5eb-2cfc92649c45 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:35.601853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 346/685] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:35.603179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165a5d0-0a11-44b0-a5eb-2cfc92649c45 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:35.612699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c8ee0-6f37-494a-87f7-f370ae576712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:35.613580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c8ee0-6f37-494a-87f7-f370ae576712 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:35.614298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c8ee0-6f37-494a-87f7-f370ae576712 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:35.614439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77c8ee0-6f37-494a-87f7-f370ae576712 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:35.625950 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77c8ee0-6f37-494a-87f7-f370ae576712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d77c8ee0-6f37-494a-87f7-f370ae576712 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:35.635415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:35.636348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:35.637162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 340/686] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:35.637858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c8ee0-6f37-494a-87f7-f370ae576712 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:35.733176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1165a5d0-0a11-44b0-a5eb-2cfc92649c45 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:35.759620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77c8ee0-6f37-494a-87f7-f370ae576712 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:35.769731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6cff9-fddc-4451-a68a-7e142832a2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:35.770488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6cff9-fddc-4451-a68a-7e142832a2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:35.770775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6cff9-fddc-4451-a68a-7e142832a2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:35.771055 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6cff9-fddc-4451-a68a-7e142832a2f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:35.808760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0583cc-4faa-4490-a3d3-977883986075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:35.809799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0583cc-4faa-4490-a3d3-977883986075 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:35.810563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0583cc-4faa-4490-a3d3-977883986075 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:35.811968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0583cc-4faa-4490-a3d3-977883986075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:35.813031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0583cc-4faa-4490-a3d3-977883986075 None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=8ff708d6758d4f7b913e763824f3b791) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:35.813573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0583cc-4faa-4490-a3d3-977883986075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:35.815273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0583cc-4faa-4490-a3d3-977883986075 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:35.825176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:35.834291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 341/687] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:35 2022] GET /identity/v3/role_assignments?user.id=8ff708d6758d4f7b913e763824f3b791&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:35.834291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0583cc-4faa-4490-a3d3-977883986075 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:35.880595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6cff9-fddc-4451-a68a-7e142832a2f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:35.883856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c6cff9-fddc-4451-a68a-7e142832a2f4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:35.923246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:36.007741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 347/688] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:36.012734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6cff9-fddc-4451-a68a-7e142832a2f4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:36.075818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0583cc-4faa-4490-a3d3-977883986075 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:36.111210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:36.112450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236993c-bc4e-43cc-a377-bf9b41625757 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:36.113223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236993c-bc4e-43cc-a377-bf9b41625757 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:36.113575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236993c-bc4e-43cc-a377-bf9b41625757 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:36.113860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236993c-bc4e-43cc-a377-bf9b41625757 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:36.115942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236993c-bc4e-43cc-a377-bf9b41625757 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:36.116276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236993c-bc4e-43cc-a377-bf9b41625757 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:36.117755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236993c-bc4e-43cc-a377-bf9b41625757 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:36.132413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 342/689] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:36.134460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236993c-bc4e-43cc-a377-bf9b41625757 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:36.142178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c6cff9-fddc-4451-a68a-7e142832a2f4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:36.185741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea39f2-d386-4d93-9287-7d03775efe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:36.186667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea39f2-d386-4d93-9287-7d03775efe47 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:36.187370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea39f2-d386-4d93-9287-7d03775efe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:36.187370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea39f2-d386-4d93-9287-7d03775efe47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:36.200102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea39f2-d386-4d93-9287-7d03775efe47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:36.201612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ea39f2-d386-4d93-9287-7d03775efe47 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b0ea39f2-d386-4d93-9287-7d03775efe47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:36.223094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:36.224010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:36.224928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:36.224928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:36.224928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:36.224928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:36.225561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 348/690] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:36.231243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea39f2-d386-4d93-9287-7d03775efe47 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:36.255379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:36.266425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236993c-bc4e-43cc-a377-bf9b41625757 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:36.307006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184155f7-fca1-40cf-827f-818f6f894144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:36.308345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184155f7-fca1-40cf-827f-818f6f894144 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:36.308628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184155f7-fca1-40cf-827f-818f6f894144 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:36.308900 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184155f7-fca1-40cf-827f-818f6f894144 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:36.326930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184155f7-fca1-40cf-827f-818f6f894144 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:36.328396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184155f7-fca1-40cf-827f-818f6f894144 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:36.350608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea39f2-d386-4d93-9287-7d03775efe47 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-184155f7-fca1-40cf-827f-818f6f894144 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:36.357712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:36.358686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:36.359558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:36.359558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:36.359558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:36.359558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:36.367136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 343/691] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:36.373497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184155f7-fca1-40cf-827f-818f6f894144 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:36.401747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:36.418208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0b004-30de-4a11-bfb4-429522657d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:36.418886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0b004-30de-4a11-bfb4-429522657d6a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:36.419170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0b004-30de-4a11-bfb4-429522657d6a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:36.419451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0b004-30de-4a11-bfb4-429522657d6a None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:36.420179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0b004-30de-4a11-bfb4-429522657d6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:36.420490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0b004-30de-4a11-bfb4-429522657d6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:36.421873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0b004-30de-4a11-bfb4-429522657d6a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:36.435197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 349/692] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:36.443032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0b004-30de-4a11-bfb4-429522657d6a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:36.552379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184155f7-fca1-40cf-827f-818f6f894144 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:36.587164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdde043-54a4-4d5e-b3e0-f1f9f80edb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:36.587859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdde043-54a4-4d5e-b3e0-f1f9f80edb5b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:36.588171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdde043-54a4-4d5e-b3e0-f1f9f80edb5b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:36.588483 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdde043-54a4-4d5e-b3e0-f1f9f80edb5b None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:36.589201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdde043-54a4-4d5e-b3e0-f1f9f80edb5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:36.589505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdde043-54a4-4d5e-b3e0-f1f9f80edb5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:36.591014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdde043-54a4-4d5e-b3e0-f1f9f80edb5b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:36.603227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 344/693] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:36.609210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdde043-54a4-4d5e-b3e0-f1f9f80edb5b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:36.618449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:36.684007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0b004-30de-4a11-bfb4-429522657d6a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:36.719388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdde043-54a4-4d5e-b3e0-f1f9f80edb5b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:36.760744 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e8e81-6df7-4314-870b-dcaa8e1e4188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:36.761868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e8e81-6df7-4314-870b-dcaa8e1e4188 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:36.763039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e8e81-6df7-4314-870b-dcaa8e1e4188 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:36.763513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85e8e81-6df7-4314-870b-dcaa8e1e4188 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:36.775077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e8e81-6df7-4314-870b-dcaa8e1e4188 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:36.776005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba513cf7-17f5-4152-a0cc-40bea751374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:36.777031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba513cf7-17f5-4152-a0cc-40bea751374c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:36.777505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba513cf7-17f5-4152-a0cc-40bea751374c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:36.777969 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba513cf7-17f5-4152-a0cc-40bea751374c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:36.780171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85e8e81-6df7-4314-870b-dcaa8e1e4188 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:36.794822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba513cf7-17f5-4152-a0cc-40bea751374c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:36.796296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba513cf7-17f5-4152-a0cc-40bea751374c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f85e8e81-6df7-4314-870b-dcaa8e1e4188 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:36.801978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:36.802969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:36.803907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:36.803907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:36.803907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:36.803907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:36.803907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:36.803907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:36.803907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:36.803907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:36.803907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:36.803907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:36.806523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 345/694] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Jun 26 02:52:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:36.812401 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e8e81-6df7-4314-870b-dcaa8e1e4188 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ba513cf7-17f5-4152-a0cc-40bea751374c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:36.816152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:36.817032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:36.817946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:36.817946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:36.817946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:36.817946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:36.817946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:36.817946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:36.817946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:36.817946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:36.826595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 350/695] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:36.827568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba513cf7-17f5-4152-a0cc-40bea751374c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:36.888766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:36.892402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:36.929894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85e8e81-6df7-4314-870b-dcaa8e1e4188 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:36.941532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba513cf7-17f5-4152-a0cc-40bea751374c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:36.967320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30206976-4599-4339-b555-700545191164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:36.968259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206976-4599-4339-b555-700545191164 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:36.968685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206976-4599-4339-b555-700545191164 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:36.969196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30206976-4599-4339-b555-700545191164 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:36.970302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30206976-4599-4339-b555-700545191164 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:36.970756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30206976-4599-4339-b555-700545191164 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:36.972456 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30206976-4599-4339-b555-700545191164 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:36.977923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7251576a-41b3-4635-8926-ed77e10af089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:36.978720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251576a-41b3-4635-8926-ed77e10af089 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:36.979160 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251576a-41b3-4635-8926-ed77e10af089 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:36.979996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7251576a-41b3-4635-8926-ed77e10af089 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:36.980871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251576a-41b3-4635-8926-ed77e10af089 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:36.981309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251576a-41b3-4635-8926-ed77e10af089 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:36.982896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7251576a-41b3-4635-8926-ed77e10af089 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:37.002067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 351/696] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:37.002067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b87e94-c912-4d79-aaba-670b9ed91066 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.002067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b87e94-c912-4d79-aaba-670b9ed91066 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.002067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b87e94-c912-4d79-aaba-670b9ed91066 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.002067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 352/697] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:37.002954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 346/698] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Jun 26 02:52:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:37.003947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b87e94-c912-4d79-aaba-670b9ed91066 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:37.004752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d7dfc-c925-49f6-9433-4d7e57f80248 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.005210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d7dfc-c925-49f6-9433-4d7e57f80248 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.005667 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d7dfc-c925-49f6-9433-4d7e57f80248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.023390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b86d7dfc-c925-49f6-9433-4d7e57f80248 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:37.024948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b86d7dfc-c925-49f6-9433-4d7e57f80248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:37.096343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 347/699] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:37.097502 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d7dfc-c925-49f6-9433-4d7e57f80248 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:37.125033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b87e94-c912-4d79-aaba-670b9ed91066 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:37.140299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:37.173758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d0054-681d-4a5f-ba2b-638eeb59c1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:37.175034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d0054-681d-4a5f-ba2b-638eeb59c1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.175034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d0054-681d-4a5f-ba2b-638eeb59c1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.175248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d0054-681d-4a5f-ba2b-638eeb59c1a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.175616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:37.207650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d7dfc-c925-49f6-9433-4d7e57f80248 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:37.246939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d0054-681d-4a5f-ba2b-638eeb59c1a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:37.248617 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94d0054-681d-4a5f-ba2b-638eeb59c1a5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:37.253010 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42ad5b-e4ad-4bdb-809e-8017bc0d1615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:37.253679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42ad5b-e4ad-4bdb-809e-8017bc0d1615 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.253961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42ad5b-e4ad-4bdb-809e-8017bc0d1615 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.254265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a42ad5b-e4ad-4bdb-809e-8017bc0d1615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.255164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42ad5b-e4ad-4bdb-809e-8017bc0d1615 None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=60f006c99809427496b026c17a663995) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:37.255513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42ad5b-e4ad-4bdb-809e-8017bc0d1615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:37.256941 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a42ad5b-e4ad-4bdb-809e-8017bc0d1615 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:37.267410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 348/700] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:37 2022] GET /identity/v3/role_assignments?user.id=60f006c99809427496b026c17a663995&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:37.268469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42ad5b-e4ad-4bdb-809e-8017bc0d1615 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:37.382994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a42ad5b-e4ad-4bdb-809e-8017bc0d1615 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:37.396654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 353/701] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:37.403478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:37.407251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d0054-681d-4a5f-ba2b-638eeb59c1a5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:37.423501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8d310-bbe2-4898-adfc-0e2167fc74b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:37.424316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8d310-bbe2-4898-adfc-0e2167fc74b4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.424599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8d310-bbe2-4898-adfc-0e2167fc74b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.424875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c8d310-bbe2-4898-adfc-0e2167fc74b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.426422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:37.437411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8d310-bbe2-4898-adfc-0e2167fc74b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:37.438916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c8d310-bbe2-4898-adfc-0e2167fc74b4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f2c8d310-bbe2-4898-adfc-0e2167fc74b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:37.459554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:37.460512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:37.461457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:37.461457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:37.461457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:37.461457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:37.461457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 349/702] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:37.466677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8d310-bbe2-4898-adfc-0e2167fc74b4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:37.542211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d0054-681d-4a5f-ba2b-638eeb59c1a5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:37.578980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f041b8-7f4a-4aeb-b8c9-72023ce76cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:37.580011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f041b8-7f4a-4aeb-b8c9-72023ce76cc2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.580410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c8d310-bbe2-4898-adfc-0e2167fc74b4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:37.580604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f041b8-7f4a-4aeb-b8c9-72023ce76cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.581020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f041b8-7f4a-4aeb-b8c9-72023ce76cc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.592348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f041b8-7f4a-4aeb-b8c9-72023ce76cc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:37.594170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f041b8-7f4a-4aeb-b8c9-72023ce76cc2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e9f041b8-7f4a-4aeb-b8c9-72023ce76cc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:37.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:37.617977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:37.618910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:37.618910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:37.618910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:37.618910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:37.618910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 354/703] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:37.620290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ca378-4303-4c14-9d55-446ce6ab7830 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.620402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ca378-4303-4c14-9d55-446ce6ab7830 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.625069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441ca378-4303-4c14-9d55-446ce6ab7830 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.625069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 355/704] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:37.626971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ca378-4303-4c14-9d55-446ce6ab7830 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:37.629819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e093b472-a58a-47a7-b750-eb7ca6099c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:37.631354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093b472-a58a-47a7-b750-eb7ca6099c04 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.631354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093b472-a58a-47a7-b750-eb7ca6099c04 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.631354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e093b472-a58a-47a7-b750-eb7ca6099c04 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.632043 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093b472-a58a-47a7-b750-eb7ca6099c04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:37.632417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093b472-a58a-47a7-b750-eb7ca6099c04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:37.634453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e093b472-a58a-47a7-b750-eb7ca6099c04 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:37.647994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 350/705] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:37.649380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dc32d-0276-4580-9327-f1e41e41349c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.649739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dc32d-0276-4580-9327-f1e41e41349c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.650125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922dc32d-0276-4580-9327-f1e41e41349c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.668434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-922dc32d-0276-4580-9327-f1e41e41349c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:37.669867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-922dc32d-0276-4580-9327-f1e41e41349c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:37.697677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:37.701084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:37.750143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 351/706] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:37.751307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dc32d-0276-4580-9327-f1e41e41349c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:37.752424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441ca378-4303-4c14-9d55-446ce6ab7830 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:37.799146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c110b1-3f6b-4839-a99f-d755d411ceb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:37.800180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c110b1-3f6b-4839-a99f-d755d411ceb3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.800535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c110b1-3f6b-4839-a99f-d755d411ceb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.800932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c110b1-3f6b-4839-a99f-d755d411ceb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.801721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c110b1-3f6b-4839-a99f-d755d411ceb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:37.802035 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c110b1-3f6b-4839-a99f-d755d411ceb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:37.803705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c110b1-3f6b-4839-a99f-d755d411ceb3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:37.820253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 356/707] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:37.821450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c110b1-3f6b-4839-a99f-d755d411ceb3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:37.890064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922dc32d-0276-4580-9327-f1e41e41349c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:37.934895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0000e4-4604-4598-98d7-61a734625985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:37.936122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0000e4-4604-4598-98d7-61a734625985 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:37.936570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0000e4-4604-4598-98d7-61a734625985 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:37.937232 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0000e4-4604-4598-98d7-61a734625985 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:37.938545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0000e4-4604-4598-98d7-61a734625985 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:37.940669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0000e4-4604-4598-98d7-61a734625985 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:37.947215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c110b1-3f6b-4839-a99f-d755d411ceb3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:37.973312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 352/708] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:52:37 2022] POST /identity/v3/users => generated 254 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:37.974939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0000e4-4604-4598-98d7-61a734625985 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:37.976239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:38.001582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f439a7-42be-43fa-9fda-98ee39363270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:38.003262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f439a7-42be-43fa-9fda-98ee39363270 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:38.005419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f439a7-42be-43fa-9fda-98ee39363270 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:38.005905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f439a7-42be-43fa-9fda-98ee39363270 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:38.006776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f439a7-42be-43fa-9fda-98ee39363270 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:38.009822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f439a7-42be-43fa-9fda-98ee39363270 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:38.015155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-08f439a7-42be-43fa-9fda-98ee39363270 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:38.045117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:38.046386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 26 02:52:38.047442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:38.058512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 357/709] 10.222.0.13 () {56 vars in 1129 bytes} [Sun Jun 26 02:52:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:38.063613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f439a7-42be-43fa-9fda-98ee39363270 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:38.169743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0000e4-4604-4598-98d7-61a734625985 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:38.190657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f439a7-42be-43fa-9fda-98ee39363270 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:38.216593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ee541-8d05-48ea-8a4d-b19157638c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:38.218002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ee541-8d05-48ea-8a4d-b19157638c15 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:38.218472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ee541-8d05-48ea-8a4d-b19157638c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:38.218915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1ee541-8d05-48ea-8a4d-b19157638c15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:38.229143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ee541-8d05-48ea-8a4d-b19157638c15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:38.231547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1ee541-8d05-48ea-8a4d-b19157638c15 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:38.235437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eed31a-cbce-484c-949f-50a0f6ed5717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:38.236114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eed31a-cbce-484c-949f-50a0f6ed5717 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:38.236404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eed31a-cbce-484c-949f-50a0f6ed5717 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:38.236682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eed31a-cbce-484c-949f-50a0f6ed5717 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:38.237396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eed31a-cbce-484c-949f-50a0f6ed5717 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:38.237706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eed31a-cbce-484c-949f-50a0f6ed5717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:38.239144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eed31a-cbce-484c-949f-50a0f6ed5717 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:38.249627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 358/710] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Jun 26 02:52:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:38.251151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eed31a-cbce-484c-949f-50a0f6ed5717 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ec1ee541-8d05-48ea-8a4d-b19157638c15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:38.252474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:38.253322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:38.254496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:38.254496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:38.254496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:38.254496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:38.254496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:38.254496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:38.254496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:38.254496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:38.254496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 353/711] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:38.263543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ee541-8d05-48ea-8a4d-b19157638c15 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:38.314875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:38.359723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:38.385946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eed31a-cbce-484c-949f-50a0f6ed5717 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:38.387289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1ee541-8d05-48ea-8a4d-b19157638c15 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:38.423333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abaab00-db65-473b-ba26-cb554260aba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:38.423927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abaab00-db65-473b-ba26-cb554260aba8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:38.424216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abaab00-db65-473b-ba26-cb554260aba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:38.424495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abaab00-db65-473b-ba26-cb554260aba8 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:38.425206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abaab00-db65-473b-ba26-cb554260aba8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:38.425518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abaab00-db65-473b-ba26-cb554260aba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:38.426985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abaab00-db65-473b-ba26-cb554260aba8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:38.434064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce3781-089b-4c1e-b843-04b16c127282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:38.434064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce3781-089b-4c1e-b843-04b16c127282 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:38.434064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce3781-089b-4c1e-b843-04b16c127282 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:38.434064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ce3781-089b-4c1e-b843-04b16c127282 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:38.434711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce3781-089b-4c1e-b843-04b16c127282 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:38.435223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce3781-089b-4c1e-b843-04b16c127282 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:38.436869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ce3781-089b-4c1e-b843-04b16c127282 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:38.438178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 359/712] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Jun 26 02:52:38 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:38.439638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa432508-564a-4a90-a3d0-6f28b9c098a4 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:38.440021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa432508-564a-4a90-a3d0-6f28b9c098a4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:38.440337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa432508-564a-4a90-a3d0-6f28b9c098a4 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:38.442031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 360/713] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:38.455621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522765ee-803f-44e0-abb4-3fa86cce72c0 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:38.455937 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522765ee-803f-44e0-abb4-3fa86cce72c0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:38.456190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522765ee-803f-44e0-abb4-3fa86cce72c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:38.457942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 354/714] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:38.464330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce3781-089b-4c1e-b843-04b16c127282 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:38.483938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-522765ee-803f-44e0-abb4-3fa86cce72c0 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:38.485161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-522765ee-803f-44e0-abb4-3fa86cce72c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:38.577599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ce3781-089b-4c1e-b843-04b16c127282 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:38.596526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 361/715] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:38.630011 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760a0a-9036-4829-ae91-824484ccfdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:38.631096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760a0a-9036-4829-ae91-824484ccfdce None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:38.631577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760a0a-9036-4829-ae91-824484ccfdce None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:38.632050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95760a0a-9036-4829-ae91-824484ccfdce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:38.633086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760a0a-9036-4829-ae91-824484ccfdce None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=9ee0150dfb3f42709029e42d0fd9105d) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:38.633598 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760a0a-9036-4829-ae91-824484ccfdce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:38.635315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95760a0a-9036-4829-ae91-824484ccfdce None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:38.638650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:38.642578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 355/716] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:38 2022] GET /identity/v3/role_assignments?user.id=9ee0150dfb3f42709029e42d0fd9105d&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:38.642734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522765ee-803f-44e0-abb4-3fa86cce72c0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:38.704295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:38.810310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522765ee-803f-44e0-abb4-3fa86cce72c0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:38.850372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5edd3b-037e-41a4-8a8a-92b98283e593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:38.851176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5edd3b-037e-41a4-8a8a-92b98283e593 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:38.851458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5edd3b-037e-41a4-8a8a-92b98283e593 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:38.851729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5edd3b-037e-41a4-8a8a-92b98283e593 None admin] PATH_INFO: `/v3/services/network` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:38.852391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5edd3b-037e-41a4-8a8a-92b98283e593 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:38.853768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5edd3b-037e-41a4-8a8a-92b98283e593 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0d5edd3b-037e-41a4-8a8a-92b98283e593 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:38.869249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:38.870565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:38.871528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 362/717] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:38 2022] GET /identity/v3/services/network => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:38.879851 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760a0a-9036-4829-ae91-824484ccfdce None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:38.983921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95760a0a-9036-4829-ae91-824484ccfdce None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:39.019170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d0135d-d435-4d19-a855-8cff063f7d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:39.019845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0135d-d435-4d19-a855-8cff063f7d0f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:39.020127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0135d-d435-4d19-a855-8cff063f7d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:39.020400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d0135d-d435-4d19-a855-8cff063f7d0f None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:39.021100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0135d-d435-4d19-a855-8cff063f7d0f None admin] RBAC: Adding query filter params (name=network) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:39.021405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0135d-d435-4d19-a855-8cff063f7d0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:39.022784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d0135d-d435-4d19-a855-8cff063f7d0f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:39.028507 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 356/718] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:52:38 2022] GET /identity/v3/services?name=network => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:39.033701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5edd3b-037e-41a4-8a8a-92b98283e593 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:39.184909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5edd3b-037e-41a4-8a8a-92b98283e593 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:39.194215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:39.220304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e151de7b-5fc9-4d33-8576-d74bdfcc5997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:39.220971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151de7b-5fc9-4d33-8576-d74bdfcc5997 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:39.221259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151de7b-5fc9-4d33-8576-d74bdfcc5997 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:39.221538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e151de7b-5fc9-4d33-8576-d74bdfcc5997 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:39.222273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151de7b-5fc9-4d33-8576-d74bdfcc5997 None admin] RBAC: Adding query filter params (type=network) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:39.222588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151de7b-5fc9-4d33-8576-d74bdfcc5997 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:39.223949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e151de7b-5fc9-4d33-8576-d74bdfcc5997 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:39.232977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 363/719] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:52:39 2022] GET /identity/v3/services?type=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:39.846292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aef735-af6d-457c-8773-42f81647adb0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:39.847007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aef735-af6d-457c-8773-42f81647adb0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:39.847007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aef735-af6d-457c-8773-42f81647adb0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:39.850417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 357/720] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:39.864283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ee8b9-7013-4f15-a74d-23623c2bdfe4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:39.864942 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ee8b9-7013-4f15-a74d-23623c2bdfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:39.864942 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ee8b9-7013-4f15-a74d-23623c2bdfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:39.887166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c64ee8b9-7013-4f15-a74d-23623c2bdfe4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:39.888421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c64ee8b9-7013-4f15-a74d-23623c2bdfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:39.986060 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 364/721] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:40.051036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aef735-af6d-457c-8773-42f81647adb0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:40.184097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:40.233679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aef735-af6d-457c-8773-42f81647adb0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:40.277563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dbb28-89d5-44f9-afc4-a17a101f8a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:40.278674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dbb28-89d5-44f9-afc4-a17a101f8a40 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:40.278960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dbb28-89d5-44f9-afc4-a17a101f8a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:40.279231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4dbb28-89d5-44f9-afc4-a17a101f8a40 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:40.294258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4dbb28-89d5-44f9-afc4-a17a101f8a40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ff4dbb28-89d5-44f9-afc4-a17a101f8a40 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:40.302808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:40.303722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:40.304582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:40.305233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 358/722] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:40.311021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ee8b9-7013-4f15-a74d-23623c2bdfe4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:40.458070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ee8b9-7013-4f15-a74d-23623c2bdfe4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:40.460014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:40.510905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffc8ca-0e26-446d-b6fa-af4f7b4906b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:40.511583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffc8ca-0e26-446d-b6fa-af4f7b4906b0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:40.511871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffc8ca-0e26-446d-b6fa-af4f7b4906b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:40.512154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ffc8ca-0e26-446d-b6fa-af4f7b4906b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:40.512909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffc8ca-0e26-446d-b6fa-af4f7b4906b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:40.513217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffc8ca-0e26-446d-b6fa-af4f7b4906b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:40.514624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ffc8ca-0e26-446d-b6fa-af4f7b4906b0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:40.525608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 365/723] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:40.530928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dbb28-89d5-44f9-afc4-a17a101f8a40 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:40.685828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4dbb28-89d5-44f9-afc4-a17a101f8a40 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:40.717103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:40.729040 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2fe14-5116-4e69-8d8f-56ce6412037d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:40.730364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2fe14-5116-4e69-8d8f-56ce6412037d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:40.730654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2fe14-5116-4e69-8d8f-56ce6412037d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:40.730931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d2fe14-5116-4e69-8d8f-56ce6412037d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:40.748109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2fe14-5116-4e69-8d8f-56ce6412037d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:40.749823 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d2fe14-5116-4e69-8d8f-56ce6412037d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c7d2fe14-5116-4e69-8d8f-56ce6412037d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:40.772960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:40.773883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:40.774864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:40.774864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:40.774864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:40.774864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:40.777020 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 359/724] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:40.781877 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffc8ca-0e26-446d-b6fa-af4f7b4906b0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:40.941199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ffc8ca-0e26-446d-b6fa-af4f7b4906b0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:40.975508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295757fa-8b1a-49a1-98ab-9cb4bb16e839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:40.976100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295757fa-8b1a-49a1-98ab-9cb4bb16e839 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:40.976391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295757fa-8b1a-49a1-98ab-9cb4bb16e839 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:40.976673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295757fa-8b1a-49a1-98ab-9cb4bb16e839 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:40.977408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295757fa-8b1a-49a1-98ab-9cb4bb16e839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:40.977722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295757fa-8b1a-49a1-98ab-9cb4bb16e839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:40.979691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295757fa-8b1a-49a1-98ab-9cb4bb16e839 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:40.997620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 366/725] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:41.003206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2fe14-5116-4e69-8d8f-56ce6412037d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:41.058145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bad96-9e00-4b80-b6d5-f3654621d648 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.058488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bad96-9e00-4b80-b6d5-f3654621d648 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.058716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6bad96-9e00-4b80-b6d5-f3654621d648 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.060376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 367/726] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:41.063570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7188d-7d21-4bab-9bac-d108d5b1d017 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.064053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7188d-7d21-4bab-9bac-d108d5b1d017 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.064600 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7188d-7d21-4bab-9bac-d108d5b1d017 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.066235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 368/727] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:41.071388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab44773-3d38-4af1-afe4-6a8d2471febc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.071924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab44773-3d38-4af1-afe4-6a8d2471febc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.072417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab44773-3d38-4af1-afe4-6a8d2471febc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.090870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cab44773-3d38-4af1-afe4-6a8d2471febc None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:41.092209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cab44773-3d38-4af1-afe4-6a8d2471febc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:41.113404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:41.146360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d2fe14-5116-4e69-8d8f-56ce6412037d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:41.186109 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 369/728] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 26 02:52:41.187745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fd8b7-db55-4613-8a55-afbab1986fb2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.188290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fd8b7-db55-4613-8a55-afbab1986fb2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.188792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2fd8b7-db55-4613-8a55-afbab1986fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.189232 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c58ed3-f1fc-483a-859a-8ae32cc484e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:41.190021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c58ed3-f1fc-483a-859a-8ae32cc484e1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.190330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c58ed3-f1fc-483a-859a-8ae32cc484e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.190604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c58ed3-f1fc-483a-859a-8ae32cc484e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.198621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c58ed3-f1fc-483a-859a-8ae32cc484e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:41.201338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c58ed3-f1fc-483a-859a-8ae32cc484e1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b7c58ed3-f1fc-483a-859a-8ae32cc484e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:41.216697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:41.217859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:41.219025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:41.219025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:41.219025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:41.219025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:41.219025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:41.219025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:41.219025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:41.219025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:41.219025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:41.219025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:41.221216 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 360/729] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:41.222309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c58ed3-f1fc-483a-859a-8ae32cc484e1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:41.223356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5b2fd8b7-db55-4613-8a55-afbab1986fb2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:41.224668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5b2fd8b7-db55-4613-8a55-afbab1986fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:41.269225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:41.297112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 370/730] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:41.298455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fd8b7-db55-4613-8a55-afbab1986fb2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:41.364403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c58ed3-f1fc-483a-859a-8ae32cc484e1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:41.392771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:41.404186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a536da-0438-4679-a472-25e22a2fa821 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:41.404859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a536da-0438-4679-a472-25e22a2fa821 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.405148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a536da-0438-4679-a472-25e22a2fa821 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.405425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a536da-0438-4679-a472-25e22a2fa821 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.406170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a536da-0438-4679-a472-25e22a2fa821 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:41.407527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a536da-0438-4679-a472-25e22a2fa821 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:41.410640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2fd8b7-db55-4613-8a55-afbab1986fb2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:41.430101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 361/731] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:52:41 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:41.430101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a536da-0438-4679-a472-25e22a2fa821 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:41.470516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d66ee6-1eae-4856-814c-5b1c1b313c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:41.471347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d66ee6-1eae-4856-814c-5b1c1b313c79 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.471631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d66ee6-1eae-4856-814c-5b1c1b313c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.471906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d66ee6-1eae-4856-814c-5b1c1b313c79 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.472756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d66ee6-1eae-4856-814c-5b1c1b313c79 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:41.473067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d66ee6-1eae-4856-814c-5b1c1b313c79 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:41.475822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d66ee6-1eae-4856-814c-5b1c1b313c79 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:41.502445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 371/732] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Jun 26 02:52:41 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:41.504247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65677cc4-bcf9-4224-81f1-085502a56f15 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.504541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65677cc4-bcf9-4224-81f1-085502a56f15 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.504817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65677cc4-bcf9-4224-81f1-085502a56f15 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.507407 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 372/733] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:41.508291 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65677cc4-bcf9-4224-81f1-085502a56f15 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:41.551357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:41.561831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a536da-0438-4679-a472-25e22a2fa821 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:41.618310 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26ee081-38db-451f-b271-7f3efbb5ee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:41.618738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee081-38db-451f-b271-7f3efbb5ee51 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.619283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee081-38db-451f-b271-7f3efbb5ee51 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.620082 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26ee081-38db-451f-b271-7f3efbb5ee51 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.633578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26ee081-38db-451f-b271-7f3efbb5ee51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:41.641519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65677cc4-bcf9-4224-81f1-085502a56f15 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-a26ee081-38db-451f-b271-7f3efbb5ee51 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:41.642934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:41.643868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:41.644692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 362/734] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:41.647281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2808d68-ef2d-49d1-954c-797db76c2c72 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.647281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2808d68-ef2d-49d1-954c-797db76c2c72 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.647281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2808d68-ef2d-49d1-954c-797db76c2c72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.666672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2808d68-ef2d-49d1-954c-797db76c2c72 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:41.668610 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2808d68-ef2d-49d1-954c-797db76c2c72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:41.688368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d33e3-2273-4302-827d-9a4b18c394c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:41.689272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d33e3-2273-4302-827d-9a4b18c394c3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.689943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d33e3-2273-4302-827d-9a4b18c394c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.690255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d33e3-2273-4302-827d-9a4b18c394c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.705562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:41.743473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 363/735] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:41 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:41.749607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2808d68-ef2d-49d1-954c-797db76c2c72 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:41.774502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d33e3-2273-4302-827d-9a4b18c394c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:41.776275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d33e3-2273-4302-827d-9a4b18c394c3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:41.887363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:41.903490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2808d68-ef2d-49d1-954c-797db76c2c72 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:41.909839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 373/736] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:41 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 26 02:52:41.914175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d33e3-2273-4302-827d-9a4b18c394c3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:41.947381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420d394-e270-48ba-85d6-5cd60475cbcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:41.948387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420d394-e270-48ba-85d6-5cd60475cbcd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.948868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420d394-e270-48ba-85d6-5cd60475cbcd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.949321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420d394-e270-48ba-85d6-5cd60475cbcd None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.950272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420d394-e270-48ba-85d6-5cd60475cbcd None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:41.950760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420d394-e270-48ba-85d6-5cd60475cbcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:41.951583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:41.952375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420d394-e270-48ba-85d6-5cd60475cbcd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:41.961185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 364/737] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:41.962368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2663b8-9a87-4c35-a4a0-2170a5f1ec83 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:41.962656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2663b8-9a87-4c35-a4a0-2170a5f1ec83 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:41.962933 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2663b8-9a87-4c35-a4a0-2170a5f1ec83 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:41.964341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 365/738] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:41.967311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2663b8-9a87-4c35-a4a0-2170a5f1ec83 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:42.020590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d33e3-2273-4302-827d-9a4b18c394c3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.058209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c24b25-3530-4c3d-b6a0-695cd68b6404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:42.059092 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c24b25-3530-4c3d-b6a0-695cd68b6404 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.059451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c24b25-3530-4c3d-b6a0-695cd68b6404 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.059801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c24b25-3530-4c3d-b6a0-695cd68b6404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.070481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c24b25-3530-4c3d-b6a0-695cd68b6404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:42.075736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2663b8-9a87-4c35-a4a0-2170a5f1ec83 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-99c24b25-3530-4c3d-b6a0-695cd68b6404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:42.078010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:42.078933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:42.079738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 374/739] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:42.080397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c24b25-3530-4c3d-b6a0-695cd68b6404 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:42.114848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90dd18-c268-4680-be3e-53c8c342662b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:42.116327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90dd18-c268-4680-be3e-53c8c342662b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.116819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90dd18-c268-4680-be3e-53c8c342662b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.117163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90dd18-c268-4680-be3e-53c8c342662b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.129885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90dd18-c268-4680-be3e-53c8c342662b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:42.131442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90dd18-c268-4680-be3e-53c8c342662b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:42.141264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2a90dd18-c268-4680-be3e-53c8c342662b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:42.151884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:42.152786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:42.153649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:42.153649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:42.153649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:42.153649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:42.156079 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 366/740] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:42.160478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65f92a-87da-453d-954d-5b98e631ba16 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.160780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65f92a-87da-453d-954d-5b98e631ba16 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.161067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65f92a-87da-453d-954d-5b98e631ba16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.178634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8d65f92a-87da-453d-954d-5b98e631ba16 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:42.179824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8d65f92a-87da-453d-954d-5b98e631ba16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:42.186848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:42.191180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c24b25-3530-4c3d-b6a0-695cd68b6404 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.224631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e2578-937f-4fcf-9b31-edf0a5a429b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:42.225424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e2578-937f-4fcf-9b31-edf0a5a429b2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.225706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e2578-937f-4fcf-9b31-edf0a5a429b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.225979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e2578-937f-4fcf-9b31-edf0a5a429b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.236524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e2578-937f-4fcf-9b31-edf0a5a429b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:42.237973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895e2578-937f-4fcf-9b31-edf0a5a429b2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:42.255080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 367/741] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:42 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-895e2578-937f-4fcf-9b31-edf0a5a429b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:42.255647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:42.256463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:42.257319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:42.257319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:42.257319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:42.257319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:42.258066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65f92a-87da-453d-954d-5b98e631ba16 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:42.261875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 375/742] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:42.262863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e2578-937f-4fcf-9b31-edf0a5a429b2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:42.371420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65f92a-87da-453d-954d-5b98e631ba16 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.372618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895e2578-937f-4fcf-9b31-edf0a5a429b2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.402168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:42.408936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5da23a-19ad-4033-bf00-7ecd4504cb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:42.409696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5da23a-19ad-4033-bf00-7ecd4504cb2c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.409983 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5da23a-19ad-4033-bf00-7ecd4504cb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.410282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5da23a-19ad-4033-bf00-7ecd4504cb2c None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.410396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c266f5-a034-49ec-af90-713223e6626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:42.410943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c266f5-a034-49ec-af90-713223e6626c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.411042 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5da23a-19ad-4033-bf00-7ecd4504cb2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:42.411227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c266f5-a034-49ec-af90-713223e6626c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.411402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5da23a-19ad-4033-bf00-7ecd4504cb2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:42.411494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c266f5-a034-49ec-af90-713223e6626c None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.412207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c266f5-a034-49ec-af90-713223e6626c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:42.412519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c266f5-a034-49ec-af90-713223e6626c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:42.412740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5da23a-19ad-4033-bf00-7ecd4504cb2c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:42.413862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c266f5-a034-49ec-af90-713223e6626c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:42.421526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 368/743] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:42.422260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5da23a-19ad-4033-bf00-7ecd4504cb2c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:42.432597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 376/744] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:42.432597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c266f5-a034-49ec-af90-713223e6626c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:42.444482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:42.529829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5da23a-19ad-4033-bf00-7ecd4504cb2c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.536855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c266f5-a034-49ec-af90-713223e6626c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.568618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf7e74-fc50-4410-9832-1a73b7c5723a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:42.569273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf7e74-fc50-4410-9832-1a73b7c5723a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.569554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf7e74-fc50-4410-9832-1a73b7c5723a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.569833 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf7e74-fc50-4410-9832-1a73b7c5723a None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.570564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf7e74-fc50-4410-9832-1a73b7c5723a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:42.570868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf7e74-fc50-4410-9832-1a73b7c5723a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:42.572214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf7e74-fc50-4410-9832-1a73b7c5723a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:42.573212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7604c-28dc-446b-939d-48244ba3daee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:42.573873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7604c-28dc-446b-939d-48244ba3daee None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.574195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7604c-28dc-446b-939d-48244ba3daee None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.574468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7604c-28dc-446b-939d-48244ba3daee None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.575192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7604c-28dc-446b-939d-48244ba3daee None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:42.576538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7604c-28dc-446b-939d-48244ba3daee None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:42.590074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 369/745] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:42.590074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf7e74-fc50-4410-9832-1a73b7c5723a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:42.598810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 377/746] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:52:42 2022] POST /identity/v3/services => generated 240 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:42.600298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7604c-28dc-446b-939d-48244ba3daee None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:42.665571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:42.686514 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:42.731586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7604c-28dc-446b-939d-48244ba3daee None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.737087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf7e74-fc50-4410-9832-1a73b7c5723a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.771318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ccb37-e3d7-4373-85fb-6c8fd94e83e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:42.772229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ccb37-e3d7-4373-85fb-6c8fd94e83e5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.772519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ccb37-e3d7-4373-85fb-6c8fd94e83e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.772797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ccb37-e3d7-4373-85fb-6c8fd94e83e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.782365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ccb37-e3d7-4373-85fb-6c8fd94e83e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:42.784031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ccb37-e3d7-4373-85fb-6c8fd94e83e5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:42.785819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7af5e5-e312-4723-abaf-e719ee0f0562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:42.786694 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7af5e5-e312-4723-abaf-e719ee0f0562 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.786975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7af5e5-e312-4723-abaf-e719ee0f0562 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.787255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7af5e5-e312-4723-abaf-e719ee0f0562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.798167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7af5e5-e312-4723-abaf-e719ee0f0562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:42.799726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7af5e5-e312-4723-abaf-e719ee0f0562 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-cf9ccb37-e3d7-4373-85fb-6c8fd94e83e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:42.800684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:42.801534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:42.802447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:42.802447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:42.802447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:42.802447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:42.802447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:42.802447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:42.802447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:42.802447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:42.802447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 378/747] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:42.802985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ccb37-e3d7-4373-85fb-6c8fd94e83e5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-ff7af5e5-e312-4723-abaf-e719ee0f0562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:42.824283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:42.825188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:42.826567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:42.826567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:42.826567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:42.826567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:42.828405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 370/748] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:42.829383 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7af5e5-e312-4723-abaf-e719ee0f0562 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:42.928521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ccb37-e3d7-4373-85fb-6c8fd94e83e5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.958534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7af5e5-e312-4723-abaf-e719ee0f0562 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:42.958939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:42.961555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:42.970233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d67f4-1e5d-4d51-9d3f-77c032a23834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:42.971361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d67f4-1e5d-4d51-9d3f-77c032a23834 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:42.971853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d67f4-1e5d-4d51-9d3f-77c032a23834 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:42.972320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d67f4-1e5d-4d51-9d3f-77c032a23834 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:42.980207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d67f4-1e5d-4d51-9d3f-77c032a23834 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:42.981988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4d67f4-1e5d-4d51-9d3f-77c032a23834 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-4d4d67f4-1e5d-4d51-9d3f-77c032a23834 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:42.998633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:42.999882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:43.000789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:43.000789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:43.000789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:43.000789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:43.000789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:43.000789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:43.000789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:43.000789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:43.000789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:52:43.000789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:43.004747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 379/749] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Jun 26 02:52:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:43.005688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d67f4-1e5d-4d51-9d3f-77c032a23834 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:43.006727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc61a9-d672-4be7-b003-ce1d767ec5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:43.007637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc61a9-d672-4be7-b003-ce1d767ec5fa None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:43.008136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc61a9-d672-4be7-b003-ce1d767ec5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:43.008902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc61a9-d672-4be7-b003-ce1d767ec5fa None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:43.011396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc61a9-d672-4be7-b003-ce1d767ec5fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:43.011718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc61a9-d672-4be7-b003-ce1d767ec5fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:43.013171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc61a9-d672-4be7-b003-ce1d767ec5fa None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:43.030406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 371/750] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:43.031314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc61a9-d672-4be7-b003-ce1d767ec5fa None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:43.114402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d67f4-1e5d-4d51-9d3f-77c032a23834 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:43.138527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc61a9-d672-4be7-b003-ce1d767ec5fa None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:43.152349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483b6f5-ad81-489f-a59b-9284b6bd06f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:43.153411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483b6f5-ad81-489f-a59b-9284b6bd06f4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:43.153892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483b6f5-ad81-489f-a59b-9284b6bd06f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:43.154564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483b6f5-ad81-489f-a59b-9284b6bd06f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:43.155488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483b6f5-ad81-489f-a59b-9284b6bd06f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:43.156015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483b6f5-ad81-489f-a59b-9284b6bd06f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:43.157694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483b6f5-ad81-489f-a59b-9284b6bd06f4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:43.173320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 380/751] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:43.174594 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483b6f5-ad81-489f-a59b-9284b6bd06f4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:43.180913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b50fe8-3114-4aad-a3dd-187f028547ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:43.181588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b50fe8-3114-4aad-a3dd-187f028547ed None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:43.181873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b50fe8-3114-4aad-a3dd-187f028547ed None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:43.182720 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b50fe8-3114-4aad-a3dd-187f028547ed None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:43.183622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b50fe8-3114-4aad-a3dd-187f028547ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:43.183936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b50fe8-3114-4aad-a3dd-187f028547ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:43.185359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b50fe8-3114-4aad-a3dd-187f028547ed None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:43.209343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 372/752] 10.222.0.13 () {56 vars in 1149 bytes} [Sun Jun 26 02:52:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:43.210267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b50fe8-3114-4aad-a3dd-187f028547ed None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:43.233725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:43.235189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:43.302438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483b6f5-ad81-489f-a59b-9284b6bd06f4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:43.323219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b50fe8-3114-4aad-a3dd-187f028547ed None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:43.339265 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b0785-c040-4b12-87b8-1829a0b15b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:43.340221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0785-c040-4b12-87b8-1829a0b15b08 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:43.340678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0785-c040-4b12-87b8-1829a0b15b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:43.341147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0785-c040-4b12-87b8-1829a0b15b08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:43.342187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b0785-c040-4b12-87b8-1829a0b15b08 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=60f006c99809427496b026c17a663995) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:43.342678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b0785-c040-4b12-87b8-1829a0b15b08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:43.344269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b0785-c040-4b12-87b8-1829a0b15b08 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:43.350573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 381/753] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:43 2022] GET /identity/v3/role_assignments?user.id=60f006c99809427496b026c17a663995&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:43.351583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b0785-c040-4b12-87b8-1829a0b15b08 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:43.375563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e400979-b038-45eb-ab5c-8d4b1e4825a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:43.376535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e400979-b038-45eb-ab5c-8d4b1e4825a9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:43.376888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e400979-b038-45eb-ab5c-8d4b1e4825a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:43.377273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e400979-b038-45eb-ab5c-8d4b1e4825a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:43.388953 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e400979-b038-45eb-ab5c-8d4b1e4825a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:43.390595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e400979-b038-45eb-ab5c-8d4b1e4825a9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0e400979-b038-45eb-ab5c-8d4b1e4825a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:43.431298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:43.432899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:43.433804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 373/754] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Jun 26 02:52:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:43.439369 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e400979-b038-45eb-ab5c-8d4b1e4825a9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:43.473796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b0785-c040-4b12-87b8-1829a0b15b08 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:43.486563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:43.513951 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5713b3-5777-4051-8aa1-e1eee4986486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:43.514941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5713b3-5777-4051-8aa1-e1eee4986486 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:43.515434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5713b3-5777-4051-8aa1-e1eee4986486 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:43.515890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5713b3-5777-4051-8aa1-e1eee4986486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:43.590718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5713b3-5777-4051-8aa1-e1eee4986486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:43.592821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5713b3-5777-4051-8aa1-e1eee4986486 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:43.598257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:43.625169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e400979-b038-45eb-ab5c-8d4b1e4825a9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:43.669292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ea389-ccbc-46f4-81bb-9e74755d5346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:43.669975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ea389-ccbc-46f4-81bb-9e74755d5346 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:43.670306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ea389-ccbc-46f4-81bb-9e74755d5346 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:43.670586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ea389-ccbc-46f4-81bb-9e74755d5346 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:43.671716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ea389-ccbc-46f4-81bb-9e74755d5346 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:43.672029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ea389-ccbc-46f4-81bb-9e74755d5346 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:43.674143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ea389-ccbc-46f4-81bb-9e74755d5346 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:43.694970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 374/755] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Jun 26 02:52:43 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:43.696792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52672-8407-47bf-b151-a707ab63f12b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:43.697286 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52672-8407-47bf-b151-a707ab63f12b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:43.697820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf52672-8407-47bf-b151-a707ab63f12b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:43.699704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 375/756] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:43.700685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf52672-8407-47bf-b151-a707ab63f12b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:43.740211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 382/757] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:52:43.741560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fadac5-7aa8-4882-93e3-4ce0d9e3be96 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:43.741840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fadac5-7aa8-4882-93e3-4ce0d9e3be96 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:43.742145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fadac5-7aa8-4882-93e3-4ce0d9e3be96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:43.767934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-89fadac5-7aa8-4882-93e3-4ce0d9e3be96 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:43.769131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-89fadac5-7aa8-4882-93e3-4ce0d9e3be96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:43.772012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:43.814632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf52672-8407-47bf-b151-a707ab63f12b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:43.846146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 383/758] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:43.847045 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fadac5-7aa8-4882-93e3-4ce0d9e3be96 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:43.857069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6133a-f411-414d-b205-b0245c86e570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:43.857892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6133a-f411-414d-b205-b0245c86e570 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:43.858206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6133a-f411-414d-b205-b0245c86e570 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:43.858484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d6133a-f411-414d-b205-b0245c86e570 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:43.878405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:43.880730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6133a-f411-414d-b205-b0245c86e570 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:43.882203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d6133a-f411-414d-b205-b0245c86e570 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-48d6133a-f411-414d-b205-b0245c86e570 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:43.900606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:43.902073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:43.902972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:43.902972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:43.902972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:43.902972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:43.902972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 376/759] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:43.906927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6133a-f411-414d-b205-b0245c86e570 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:43.980815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fadac5-7aa8-4882-93e3-4ce0d9e3be96 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.009899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d6133a-f411-414d-b205-b0245c86e570 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.016902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24b9d4-d080-4825-b8db-c20c8f203b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.017997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24b9d4-d080-4825-b8db-c20c8f203b55 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.018559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24b9d4-d080-4825-b8db-c20c8f203b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.019062 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24b9d4-d080-4825-b8db-c20c8f203b55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.028421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:44.031437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24b9d4-d080-4825-b8db-c20c8f203b55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.033188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a24b9d4-d080-4825-b8db-c20c8f203b55 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.045903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711dad3-72c3-44ca-b78c-7715c541109f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.046730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711dad3-72c3-44ca-b78c-7715c541109f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.047013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711dad3-72c3-44ca-b78c-7715c541109f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.047286 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711dad3-72c3-44ca-b78c-7715c541109f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.047990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711dad3-72c3-44ca-b78c-7715c541109f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.049330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711dad3-72c3-44ca-b78c-7715c541109f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5a24b9d4-d080-4825-b8db-c20c8f203b55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:44.054552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:44.055428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:44.056293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:44.056293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:44.056293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:44.056293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:44.062815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 384/760] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b711dad3-72c3-44ca-b78c-7715c541109f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:44.063460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:44.064348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 26 02:52:44.065175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:44.067487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24b9d4-d080-4825-b8db-c20c8f203b55 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.070148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 377/761] 10.222.0.13 () {56 vars in 1129 bytes} [Sun Jun 26 02:52:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:44.070972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711dad3-72c3-44ca-b78c-7715c541109f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.129523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:44.176360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24b9d4-d080-4825-b8db-c20c8f203b55 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.176561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711dad3-72c3-44ca-b78c-7715c541109f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.211555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c4ed-4330-41bb-9bc8-83461bc47f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.212246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c4ed-4330-41bb-9bc8-83461bc47f00 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.212529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c4ed-4330-41bb-9bc8-83461bc47f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.212805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff4c4ed-4330-41bb-9bc8-83461bc47f00 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.214104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c4ed-4330-41bb-9bc8-83461bc47f00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:44.214424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c4ed-4330-41bb-9bc8-83461bc47f00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.215180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a3e66-1b5c-4f86-8dff-43b7b01cd4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.215786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff4c4ed-4330-41bb-9bc8-83461bc47f00 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.216334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a3e66-1b5c-4f86-8dff-43b7b01cd4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.216815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a3e66-1b5c-4f86-8dff-43b7b01cd4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.217286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0a3e66-1b5c-4f86-8dff-43b7b01cd4c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.218241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a3e66-1b5c-4f86-8dff-43b7b01cd4c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:44.218825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a3e66-1b5c-4f86-8dff-43b7b01cd4c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.220378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0a3e66-1b5c-4f86-8dff-43b7b01cd4c5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.232647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 378/762] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:44.232647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c4ed-4330-41bb-9bc8-83461bc47f00 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.237938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 385/763] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:44.239084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a3e66-1b5c-4f86-8dff-43b7b01cd4c5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.295033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:44.332375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff4c4ed-4330-41bb-9bc8-83461bc47f00 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.347078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0a3e66-1b5c-4f86-8dff-43b7b01cd4c5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.349774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:44.369026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5687c-dfc4-42df-9d1d-7b6bd1f5ff20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.369971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5687c-dfc4-42df-9d1d-7b6bd1f5ff20 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.370420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5687c-dfc4-42df-9d1d-7b6bd1f5ff20 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.370837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de5687c-dfc4-42df-9d1d-7b6bd1f5ff20 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.371716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5687c-dfc4-42df-9d1d-7b6bd1f5ff20 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:44.372197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5687c-dfc4-42df-9d1d-7b6bd1f5ff20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.373737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de5687c-dfc4-42df-9d1d-7b6bd1f5ff20 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.383288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 379/764] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:44.384342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5687c-dfc4-42df-9d1d-7b6bd1f5ff20 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.385045 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670ca31-c4e8-41b8-adba-7acc7dfd14b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.386494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670ca31-c4e8-41b8-adba-7acc7dfd14b8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.386849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670ca31-c4e8-41b8-adba-7acc7dfd14b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.387201 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0670ca31-c4e8-41b8-adba-7acc7dfd14b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.394887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670ca31-c4e8-41b8-adba-7acc7dfd14b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.396440 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0670ca31-c4e8-41b8-adba-7acc7dfd14b8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0670ca31-c4e8-41b8-adba-7acc7dfd14b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:44.413077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:44.414003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:44.415203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:44.415203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:44.415203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:44.415203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:44.415203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:44.415203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:44.415203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:44.415203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:44.415203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 386/765] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:44.419935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670ca31-c4e8-41b8-adba-7acc7dfd14b8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.496978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de5687c-dfc4-42df-9d1d-7b6bd1f5ff20 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.525285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0670ca31-c4e8-41b8-adba-7acc7dfd14b8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.534032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6875ef-4bc8-4289-a155-8e6cad22434b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.535122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6875ef-4bc8-4289-a155-8e6cad22434b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.535625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6875ef-4bc8-4289-a155-8e6cad22434b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.536094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6875ef-4bc8-4289-a155-8e6cad22434b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.539210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:44.543890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6875ef-4bc8-4289-a155-8e6cad22434b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.545614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6875ef-4bc8-4289-a155-8e6cad22434b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6c6875ef-4bc8-4289-a155-8e6cad22434b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:44.561740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:44.562808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:44.563677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:44.563677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:44.563677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:44.563677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:44.563677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:44.563677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:44.563677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:44.563677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:44.565193 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84add399-ea6f-405f-8db2-c0f8b24ab41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.565866 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add399-ea6f-405f-8db2-c0f8b24ab41e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.566182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add399-ea6f-405f-8db2-c0f8b24ab41e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.566468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84add399-ea6f-405f-8db2-c0f8b24ab41e None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.567088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 380/766] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:44.567198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84add399-ea6f-405f-8db2-c0f8b24ab41e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:44.567473 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84add399-ea6f-405f-8db2-c0f8b24ab41e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.568474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6875ef-4bc8-4289-a155-8e6cad22434b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.568957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84add399-ea6f-405f-8db2-c0f8b24ab41e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.579837 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 387/767] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:44.579837 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84add399-ea6f-405f-8db2-c0f8b24ab41e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.619258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:44.683914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6875ef-4bc8-4289-a155-8e6cad22434b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.684109 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84add399-ea6f-405f-8db2-c0f8b24ab41e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.720272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dea4b-0c4f-4142-89cd-f8d7fcc79415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.720754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06770f-e1c5-43e8-93b8-8ce54d99fa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.720975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dea4b-0c4f-4142-89cd-f8d7fcc79415 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.721215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dea4b-0c4f-4142-89cd-f8d7fcc79415 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.721493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dea4b-0c4f-4142-89cd-f8d7fcc79415 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.722249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06770f-e1c5-43e8-93b8-8ce54d99fa7e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.722355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dea4b-0c4f-4142-89cd-f8d7fcc79415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:44.722659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dea4b-0c4f-4142-89cd-f8d7fcc79415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.723046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06770f-e1c5-43e8-93b8-8ce54d99fa7e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.723507 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06770f-e1c5-43e8-93b8-8ce54d99fa7e None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.724099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dea4b-0c4f-4142-89cd-f8d7fcc79415 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.724688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06770f-e1c5-43e8-93b8-8ce54d99fa7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:44.725205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06770f-e1c5-43e8-93b8-8ce54d99fa7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.726845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06770f-e1c5-43e8-93b8-8ce54d99fa7e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.737076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 388/768] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:44.737835 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dea4b-0c4f-4142-89cd-f8d7fcc79415 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.741324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 381/769] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:44.742448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06770f-e1c5-43e8-93b8-8ce54d99fa7e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.781280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:44.846142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:44.849663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06770f-e1c5-43e8-93b8-8ce54d99fa7e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.852403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dea4b-0c4f-4142-89cd-f8d7fcc79415 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:44.887371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288886e5-4f3a-4857-a353-d30c5c82cbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.888256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288886e5-4f3a-4857-a353-d30c5c82cbd0 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.888588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288886e5-4f3a-4857-a353-d30c5c82cbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.888802 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288886e5-4f3a-4857-a353-d30c5c82cbd0 None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/9ee0150dfb3f42709029e42d0fd9105d/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.890726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288886e5-4f3a-4857-a353-d30c5c82cbd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=9ee0150dfb3f42709029e42d0fd9105d, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.890943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9f3ab-dd71-4bee-8cbf-874ff975c724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:44.891602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9f3ab-dd71-4bee-8cbf-874ff975c724 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.891885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9f3ab-dd71-4bee-8cbf-874ff975c724 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.892140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288886e5-4f3a-4857-a353-d30c5c82cbd0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.892233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9f3ab-dd71-4bee-8cbf-874ff975c724 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.892935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9f3ab-dd71-4bee-8cbf-874ff975c724 None admin] RBAC: Adding query filter params (interface=public, service_id=e0c8cf6ffbef40bda6b1a5433975b8aa, region_id=RegionOne) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:44.893248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9f3ab-dd71-4bee-8cbf-874ff975c724 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:44.894622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9f3ab-dd71-4bee-8cbf-874ff975c724 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:44.903269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 389/770] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/endpoints?service_id=e0c8cf6ffbef40bda6b1a5433975b8aa&interface=public®ion_id=RegionOne => generated 194 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:44.903898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9f3ab-dd71-4bee-8cbf-874ff975c724 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:44.918086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 382/771] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:52:44 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/9ee0150dfb3f42709029e42d0fd9105d/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:44.918086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96772e5d-6595-49f0-aa44-626424fb9b9f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:44.918086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96772e5d-6595-49f0-aa44-626424fb9b9f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:44.918086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96772e5d-6595-49f0-aa44-626424fb9b9f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:44.918086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 383/772] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:44.918086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96772e5d-6595-49f0-aa44-626424fb9b9f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:45.036731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9f3ab-dd71-4bee-8cbf-874ff975c724 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:45.038546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:45.068831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96772e5d-6595-49f0-aa44-626424fb9b9f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:45.074906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7256b875-b640-487e-8f32-10042b523287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:45.075574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256b875-b640-487e-8f32-10042b523287 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:45.075857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256b875-b640-487e-8f32-10042b523287 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:45.076186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7256b875-b640-487e-8f32-10042b523287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:45.077034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256b875-b640-487e-8f32-10042b523287 None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=828fd11b79d045dc85e0973cd255bcf7) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:45.077346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256b875-b640-487e-8f32-10042b523287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:45.078925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7256b875-b640-487e-8f32-10042b523287 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:45.085545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 390/773] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/role_assignments?user.id=828fd11b79d045dc85e0973cd255bcf7&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:45.087027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dba50-3a79-4050-9630-21320f07a8a5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:45.087320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dba50-3a79-4050-9630-21320f07a8a5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:45.087599 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590dba50-3a79-4050-9630-21320f07a8a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:45.117336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-590dba50-3a79-4050-9630-21320f07a8a5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:45.118988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-590dba50-3a79-4050-9630-21320f07a8a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:45.124731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb1d11-c03f-4e4c-b40c-01c18dda38d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:45.125406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb1d11-c03f-4e4c-b40c-01c18dda38d6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:45.125687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb1d11-c03f-4e4c-b40c-01c18dda38d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:45.125963 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb1d11-c03f-4e4c-b40c-01c18dda38d6 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:45.126734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb1d11-c03f-4e4c-b40c-01c18dda38d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:45.128189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb1d11-c03f-4e4c-b40c-01c18dda38d6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:45.137201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 384/774] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Jun 26 02:52:44 2022] GET /identity/v3/services => generated 772 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:45.209500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 391/775] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:45.234553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb1d11-c03f-4e4c-b40c-01c18dda38d6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:45.275621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:45.366789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb1d11-c03f-4e4c-b40c-01c18dda38d6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:45.405872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3a9151-05ad-4f19-9811-907008ce64ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:45.406713 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a9151-05ad-4f19-9811-907008ce64ea None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:45.406994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a9151-05ad-4f19-9811-907008ce64ea None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:45.407269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a9151-05ad-4f19-9811-907008ce64ea None admin] PATH_INFO: `/v3/services/network` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:45.407921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a9151-05ad-4f19-9811-907008ce64ea None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:45.409279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3a9151-05ad-4f19-9811-907008ce64ea None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8e3a9151-05ad-4f19-9811-907008ce64ea None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:45.421617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:45.422661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:45.423464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 385/776] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:45 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:45.428575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dba50-3a79-4050-9630-21320f07a8a5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:45.541302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a739-8f92-4324-a665-0e9aa5c4a53a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:45.542218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a739-8f92-4324-a665-0e9aa5c4a53a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:45.542401 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f3a739-8f92-4324-a665-0e9aa5c4a53a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:45.543901 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 386/777] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:45.549836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8a767-2b68-4c44-9566-5fab2b8a8a78 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:45.550299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8a767-2b68-4c44-9566-5fab2b8a8a78 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:45.550730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f8a767-2b68-4c44-9566-5fab2b8a8a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:45.583189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-19f8a767-2b68-4c44-9566-5fab2b8a8a78 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:45.584393 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-19f8a767-2b68-4c44-9566-5fab2b8a8a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:45.589184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:45.600967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590dba50-3a79-4050-9630-21320f07a8a5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:45.667352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb452993-3633-4b48-8891-c1955cb8ebf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:45.668027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb452993-3633-4b48-8891-c1955cb8ebf5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:45.668314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb452993-3633-4b48-8891-c1955cb8ebf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:45.668588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb452993-3633-4b48-8891-c1955cb8ebf5 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:45.669303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb452993-3633-4b48-8891-c1955cb8ebf5 None admin] RBAC: Adding query filter params (name=network) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:45.669608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb452993-3633-4b48-8891-c1955cb8ebf5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:45.671027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb452993-3633-4b48-8891-c1955cb8ebf5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:45.680400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 392/778] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:52:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:45.686742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb452993-3633-4b48-8891-c1955cb8ebf5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:45.717791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 387/779] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:45.743900 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8a767-2b68-4c44-9566-5fab2b8a8a78 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:45.790068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:45.862437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f8a767-2b68-4c44-9566-5fab2b8a8a78 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:45.904258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb452993-3633-4b48-8891-c1955cb8ebf5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:45.921293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de1fc0-ad72-4ead-a628-2d31c0d84db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:45.922292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de1fc0-ad72-4ead-a628-2d31c0d84db6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:45.922420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de1fc0-ad72-4ead-a628-2d31c0d84db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:45.922696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de1fc0-ad72-4ead-a628-2d31c0d84db6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:45.941814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de1fc0-ad72-4ead-a628-2d31c0d84db6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-50de1fc0-ad72-4ead-a628-2d31c0d84db6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:45.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:45.951758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:45.952580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:45.953213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 388/780] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:45.959524 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de1fc0-ad72-4ead-a628-2d31c0d84db6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:45.982255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcf0fc-c592-4024-ad5a-9f59449cf2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:45.983984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcf0fc-c592-4024-ad5a-9f59449cf2ca None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:45.984273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcf0fc-c592-4024-ad5a-9f59449cf2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:45.984865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febcf0fc-c592-4024-ad5a-9f59449cf2ca None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:45.986913 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcf0fc-c592-4024-ad5a-9f59449cf2ca None admin] RBAC: Adding query filter params (type=network) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:45.987234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcf0fc-c592-4024-ad5a-9f59449cf2ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:45.989336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febcf0fc-c592-4024-ad5a-9f59449cf2ca None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:46.001026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 393/781] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:52:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:46.008803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcf0fc-c592-4024-ad5a-9f59449cf2ca None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:46.056013 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:46.113807 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:46.165697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de1fc0-ad72-4ead-a628-2d31c0d84db6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:46.202425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c59960-7223-40fe-a2a3-1d0b539e0692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:46.202752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febcf0fc-c592-4024-ad5a-9f59449cf2ca None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:46.203015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c59960-7223-40fe-a2a3-1d0b539e0692 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:46.203295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c59960-7223-40fe-a2a3-1d0b539e0692 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:46.203564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c59960-7223-40fe-a2a3-1d0b539e0692 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:46.204475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c59960-7223-40fe-a2a3-1d0b539e0692 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:46.204781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c59960-7223-40fe-a2a3-1d0b539e0692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:46.206176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c59960-7223-40fe-a2a3-1d0b539e0692 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:46.216317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 389/782] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:46.221690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c59960-7223-40fe-a2a3-1d0b539e0692 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:46.289361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05fca4-c0d3-40bf-9aec-63cbe3ab68ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:46.290197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05fca4-c0d3-40bf-9aec-63cbe3ab68ea None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:46.290484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05fca4-c0d3-40bf-9aec-63cbe3ab68ea None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:46.290761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f05fca4-c0d3-40bf-9aec-63cbe3ab68ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:46.292106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05fca4-c0d3-40bf-9aec-63cbe3ab68ea None admin] RBAC: Adding query filter params (interface=public, service_id=88066838d8884a4a8b85f3b400382dbf, region_id=RegionOne) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:46.292421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05fca4-c0d3-40bf-9aec-63cbe3ab68ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:46.293903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f05fca4-c0d3-40bf-9aec-63cbe3ab68ea None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:46.306655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 394/783] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Jun 26 02:52:46 2022] GET /identity/v3/endpoints?service_id=88066838d8884a4a8b85f3b400382dbf&interface=public®ion_id=RegionOne => generated 194 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:46.312072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05fca4-c0d3-40bf-9aec-63cbe3ab68ea None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:46.336218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c59960-7223-40fe-a2a3-1d0b539e0692 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:46.377873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8074d9-daf9-40f8-bb42-089f1c5493da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:46.378707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8074d9-daf9-40f8-bb42-089f1c5493da None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:46.378989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8074d9-daf9-40f8-bb42-089f1c5493da None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:46.379270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8074d9-daf9-40f8-bb42-089f1c5493da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:46.398750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:46.402729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8074d9-daf9-40f8-bb42-089f1c5493da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:46.404353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8074d9-daf9-40f8-bb42-089f1c5493da None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5d8074d9-daf9-40f8-bb42-089f1c5493da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:46.435241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:46.437326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:46.439465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:46.439465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:46.439465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:46.439465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:46.446483 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 390/784] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:46.452376 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8074d9-daf9-40f8-bb42-089f1c5493da None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:46.461716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:46.481154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f05fca4-c0d3-40bf-9aec-63cbe3ab68ea None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:46.520780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553af1af-3038-43b1-8fd9-39e26e7d2823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:46.521919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553af1af-3038-43b1-8fd9-39e26e7d2823 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:46.522647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553af1af-3038-43b1-8fd9-39e26e7d2823 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:46.523097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553af1af-3038-43b1-8fd9-39e26e7d2823 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:46.524218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553af1af-3038-43b1-8fd9-39e26e7d2823 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:46.525672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553af1af-3038-43b1-8fd9-39e26e7d2823 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:46.532840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 395/785] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Jun 26 02:52:46 2022] GET /identity/v3/services => generated 772 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:46.614989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8074d9-daf9-40f8-bb42-089f1c5493da None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:46.652873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80e7b0-315d-436d-8f54-a1481f7195a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:46.653561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80e7b0-315d-436d-8f54-a1481f7195a9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:46.653844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80e7b0-315d-436d-8f54-a1481f7195a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:46.654148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80e7b0-315d-436d-8f54-a1481f7195a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:46.655001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80e7b0-315d-436d-8f54-a1481f7195a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:46.655311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80e7b0-315d-436d-8f54-a1481f7195a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:46.656697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80e7b0-315d-436d-8f54-a1481f7195a9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:46.670533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 391/786] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:46.679167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553af1af-3038-43b1-8fd9-39e26e7d2823 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:46.792271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553af1af-3038-43b1-8fd9-39e26e7d2823 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:46.826640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b7187-4aaf-4414-ba14-21b86e92b41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:46.827430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b7187-4aaf-4414-ba14-21b86e92b41a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:46.827712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b7187-4aaf-4414-ba14-21b86e92b41a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:46.827998 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9b7187-4aaf-4414-ba14-21b86e92b41a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:46.835257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b7187-4aaf-4414-ba14-21b86e92b41a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:46.836684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9b7187-4aaf-4414-ba14-21b86e92b41a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-fa9b7187-4aaf-4414-ba14-21b86e92b41a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:46.851871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:46.852958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:46.854533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 396/787] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:46.859868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80e7b0-315d-436d-8f54-a1481f7195a9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:46.919666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:46.967134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80e7b0-315d-436d-8f54-a1481f7195a9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:47.002315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18f2fb-ef10-4af4-bae4-86d88615e03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:47.002987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18f2fb-ef10-4af4-bae4-86d88615e03f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.003270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18f2fb-ef10-4af4-bae4-86d88615e03f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.003544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18f2fb-ef10-4af4-bae4-86d88615e03f None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.004370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18f2fb-ef10-4af4-bae4-86d88615e03f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:47.004682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18f2fb-ef10-4af4-bae4-86d88615e03f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:47.006110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18f2fb-ef10-4af4-bae4-86d88615e03f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:47.023922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 392/788] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Jun 26 02:52:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:47.035132 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b7187-4aaf-4414-ba14-21b86e92b41a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:47.048330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:47.168545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9b7187-4aaf-4414-ba14-21b86e92b41a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:47.217612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075317c9-98fe-427b-8dd3-c4e05f874a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:47.218435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075317c9-98fe-427b-8dd3-c4e05f874a23 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.218720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075317c9-98fe-427b-8dd3-c4e05f874a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.219020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075317c9-98fe-427b-8dd3-c4e05f874a23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.235046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075317c9-98fe-427b-8dd3-c4e05f874a23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:47.236489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075317c9-98fe-427b-8dd3-c4e05f874a23 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-075317c9-98fe-427b-8dd3-c4e05f874a23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:47.265646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:47.266712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:47.267586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:47.267586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:47.267586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:47.267586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:47.271300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 397/789] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:47.277104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18f2fb-ef10-4af4-bae4-86d88615e03f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:47.393299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18f2fb-ef10-4af4-bae4-86d88615e03f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:47.418554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:47.446965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae47d72e-22f0-4f07-bbd5-c8fdc9800b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:47.448192 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47d72e-22f0-4f07-bbd5-c8fdc9800b78 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.448476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47d72e-22f0-4f07-bbd5-c8fdc9800b78 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.448754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae47d72e-22f0-4f07-bbd5-c8fdc9800b78 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.449471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47d72e-22f0-4f07-bbd5-c8fdc9800b78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:47.449777 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47d72e-22f0-4f07-bbd5-c8fdc9800b78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:47.451162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae47d72e-22f0-4f07-bbd5-c8fdc9800b78 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:47.467099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 393/790] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:47.475139 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075317c9-98fe-427b-8dd3-c4e05f874a23 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:47.541408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05fc8e-6183-4fed-a3d6-494a78f2fabc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.541733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05fc8e-6183-4fed-a3d6-494a78f2fabc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.541977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05fc8e-6183-4fed-a3d6-494a78f2fabc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.543615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 394/791] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:47.548611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6eb2d-7037-4995-afc1-619259f896a3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.548990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6eb2d-7037-4995-afc1-619259f896a3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.549164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6eb2d-7037-4995-afc1-619259f896a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.578406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:47.578598 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-14a6eb2d-7037-4995-afc1-619259f896a3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:47.579670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-14a6eb2d-7037-4995-afc1-619259f896a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:47.627624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075317c9-98fe-427b-8dd3-c4e05f874a23 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:47.658469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 395/792] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:52:47.659814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61fefd-ae31-4153-9380-8b2a3855f41d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.660108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61fefd-ae31-4153-9380-8b2a3855f41d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.660381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61fefd-ae31-4153-9380-8b2a3855f41d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.662028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 396/793] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:47.671977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a8915-4c69-45e6-9253-b75567473905 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.672260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a8915-4c69-45e6-9253-b75567473905 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.672534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a8915-4c69-45e6-9253-b75567473905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.673594 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab27fc-f084-4222-b465-d41bb13dbc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:47.674474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab27fc-f084-4222-b465-d41bb13dbc0c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.674762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab27fc-f084-4222-b465-d41bb13dbc0c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.675046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab27fc-f084-4222-b465-d41bb13dbc0c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.683294 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab27fc-f084-4222-b465-d41bb13dbc0c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:47.684850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab27fc-f084-4222-b465-d41bb13dbc0c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:47.697113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b99a8915-4c69-45e6-9253-b75567473905 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:47.698861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b99a8915-4c69-45e6-9253-b75567473905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-17ab27fc-f084-4222-b465-d41bb13dbc0c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:47.699943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:47.700799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:47.701671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:47.701671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:47.701671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:47.701671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:47.701671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:47.701671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:47.701671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:47.701671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:47.701671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 398/794] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:47.702606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab27fc-f084-4222-b465-d41bb13dbc0c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:47.770205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 397/795] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:47.771584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a8915-4c69-45e6-9253-b75567473905 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:47.806557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab27fc-f084-4222-b465-d41bb13dbc0c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:47.816869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:47.832935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:47.846349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476ca22-e2b6-4b6f-8bd1-ca9b701e0862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:47.847140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476ca22-e2b6-4b6f-8bd1-ca9b701e0862 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.847422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476ca22-e2b6-4b6f-8bd1-ca9b701e0862 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.847696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476ca22-e2b6-4b6f-8bd1-ca9b701e0862 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.860963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476ca22-e2b6-4b6f-8bd1-ca9b701e0862 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e476ca22-e2b6-4b6f-8bd1-ca9b701e0862 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:47.868577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:47.870079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:47.870933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 399/796] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:47.871617 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b15a8-9d70-4d65-b575-477b6d570c3a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.872440 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b15a8-9d70-4d65-b575-477b6d570c3a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.872440 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b15a8-9d70-4d65-b575-477b6d570c3a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.877613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 400/797] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:47.877613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b15a8-9d70-4d65-b575-477b6d570c3a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:47.899069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a8915-4c69-45e6-9253-b75567473905 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:47.935921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f877e0-c69a-401a-97f5-9952d244417b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:47.936866 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f877e0-c69a-401a-97f5-9952d244417b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:47.937259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f877e0-c69a-401a-97f5-9952d244417b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:47.937635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f877e0-c69a-401a-97f5-9952d244417b None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:47.938716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f877e0-c69a-401a-97f5-9952d244417b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:47.939041 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f877e0-c69a-401a-97f5-9952d244417b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:47.941054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f877e0-c69a-401a-97f5-9952d244417b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:47.959417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 398/798] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:47.960093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f877e0-c69a-401a-97f5-9952d244417b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:47.984742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b15a8-9d70-4d65-b575-477b6d570c3a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:48.021932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754fcc1-7f03-44cb-958b-8afe9f24fdcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:48.023554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754fcc1-7f03-44cb-958b-8afe9f24fdcc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.024026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754fcc1-7f03-44cb-958b-8afe9f24fdcc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.024660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754fcc1-7f03-44cb-958b-8afe9f24fdcc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.038155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754fcc1-7f03-44cb-958b-8afe9f24fdcc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-c754fcc1-7f03-44cb-958b-8afe9f24fdcc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:48.048819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:48.049794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:48.050643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:48.052272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 401/799] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:48.052990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313730e-7bc5-431a-af06-9ba57c40b422 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.053304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313730e-7bc5-431a-af06-9ba57c40b422 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.053585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a313730e-7bc5-431a-af06-9ba57c40b422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.060620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:48.071870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f877e0-c69a-401a-97f5-9952d244417b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:48.072054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a313730e-7bc5-431a-af06-9ba57c40b422 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:48.073366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a313730e-7bc5-431a-af06-9ba57c40b422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:48.108805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:48.110827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa680c13-a59b-45d3-b9aa-195ee565dc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:48.111606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa680c13-a59b-45d3-b9aa-195ee565dc6e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.111903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa680c13-a59b-45d3-b9aa-195ee565dc6e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.112184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa680c13-a59b-45d3-b9aa-195ee565dc6e None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.112932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa680c13-a59b-45d3-b9aa-195ee565dc6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:48.113237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa680c13-a59b-45d3-b9aa-195ee565dc6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:48.114920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa680c13-a59b-45d3-b9aa-195ee565dc6e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:48.129112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 399/800] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:48.129771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa680c13-a59b-45d3-b9aa-195ee565dc6e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:48.150061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 402/801] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:48.150061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313730e-7bc5-431a-af06-9ba57c40b422 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:48.251409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa680c13-a59b-45d3-b9aa-195ee565dc6e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:48.258209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a313730e-7bc5-431a-af06-9ba57c40b422 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:48.288367 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef108030-82e4-44b2-b03b-af93fc2312b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:48.289341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108030-82e4-44b2-b03b-af93fc2312b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.289623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108030-82e4-44b2-b03b-af93fc2312b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.289895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef108030-82e4-44b2-b03b-af93fc2312b6 None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/60f006c99809427496b026c17a663995/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.292544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef108030-82e4-44b2-b03b-af93fc2312b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=60f006c99809427496b026c17a663995, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:48.293965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef108030-82e4-44b2-b03b-af93fc2312b6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:48.304451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1030cd1-39f0-4f00-a7cd-6e56c274c0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:48.305251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1030cd1-39f0-4f00-a7cd-6e56c274c0ac None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.305638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1030cd1-39f0-4f00-a7cd-6e56c274c0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.306001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1030cd1-39f0-4f00-a7cd-6e56c274c0ac None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.306903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1030cd1-39f0-4f00-a7cd-6e56c274c0ac None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:48.307285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1030cd1-39f0-4f00-a7cd-6e56c274c0ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:48.318388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1030cd1-39f0-4f00-a7cd-6e56c274c0ac None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:48.318593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 400/802] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:52:48 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/60f006c99809427496b026c17a663995/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:48.318593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef108030-82e4-44b2-b03b-af93fc2312b6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:48.327510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 403/803] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:48.328536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1030cd1-39f0-4f00-a7cd-6e56c274c0ac None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:48.335692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:48.370114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:48.444911 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef108030-82e4-44b2-b03b-af93fc2312b6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:48.460812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1030cd1-39f0-4f00-a7cd-6e56c274c0ac None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:48.492995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8b025-66fc-49bc-b0fb-32d4e7f60555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:48.493805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8b025-66fc-49bc-b0fb-32d4e7f60555 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.494437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8b025-66fc-49bc-b0fb-32d4e7f60555 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.494716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d8b025-66fc-49bc-b0fb-32d4e7f60555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.508140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8b025-66fc-49bc-b0fb-32d4e7f60555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:48.509527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d8b025-66fc-49bc-b0fb-32d4e7f60555 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:48.509632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653750e-925f-45aa-8616-6e9746fe6524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:48.510482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653750e-925f-45aa-8616-6e9746fe6524 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.510770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653750e-925f-45aa-8616-6e9746fe6524 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.511051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f653750e-925f-45aa-8616-6e9746fe6524 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.511702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653750e-925f-45aa-8616-6e9746fe6524 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:48.513056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f653750e-925f-45aa-8616-6e9746fe6524 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f653750e-925f-45aa-8616-6e9746fe6524 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:48.526172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:48.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:48.527912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 404/804] 10.222.0.13 () {56 vars in 1129 bytes} [Sun Jun 26 02:52:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-68d8b025-66fc-49bc-b0fb-32d4e7f60555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:48.528787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:48.529560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:48.531800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:48.531800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:48.531800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:48.531800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:48.532334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653750e-925f-45aa-8616-6e9746fe6524 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:48.532480 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 401/805] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:48.535963 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8b025-66fc-49bc-b0fb-32d4e7f60555 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:48.624188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:48.647063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:48.651775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f653750e-925f-45aa-8616-6e9746fe6524 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:48.669196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d8b025-66fc-49bc-b0fb-32d4e7f60555 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:48.700343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a2ad5-5fcd-4fa5-984a-ed9568ca19e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:48.701136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a2ad5-5fcd-4fa5-984a-ed9568ca19e3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.701415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a2ad5-5fcd-4fa5-984a-ed9568ca19e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.701683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a2ad5-5fcd-4fa5-984a-ed9568ca19e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.710217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b1901-06fc-41b5-aea6-e323f2434d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:48.711288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b1901-06fc-41b5-aea6-e323f2434d03 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.711779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b1901-06fc-41b5-aea6-e323f2434d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.712254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4b1901-06fc-41b5-aea6-e323f2434d03 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.713813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b1901-06fc-41b5-aea6-e323f2434d03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:48.713930 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a2ad5-5fcd-4fa5-984a-ed9568ca19e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:48.714455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b1901-06fc-41b5-aea6-e323f2434d03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:48.715402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a2ad5-5fcd-4fa5-984a-ed9568ca19e3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:48.716448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4b1901-06fc-41b5-aea6-e323f2434d03 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:48.729847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 402/806] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:48.731071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b1901-06fc-41b5-aea6-e323f2434d03 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e89a2ad5-5fcd-4fa5-984a-ed9568ca19e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:48.733243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:48.734130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:48.734999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:48.734999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:48.734999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:48.734999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:48.734999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 405/807] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:48.735695 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a2ad5-5fcd-4fa5-984a-ed9568ca19e3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:48.842491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4b1901-06fc-41b5-aea6-e323f2434d03 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:48.842798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a2ad5-5fcd-4fa5-984a-ed9568ca19e3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:48.881463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:48.883215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64821c-8523-4c21-8b9c-29c6c25df6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:48.884216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64821c-8523-4c21-8b9c-29c6c25df6cf None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.884502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64821c-8523-4c21-8b9c-29c6c25df6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.884828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64821c-8523-4c21-8b9c-29c6c25df6cf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.894350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64821c-8523-4c21-8b9c-29c6c25df6cf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:48.894539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a66b8d-79d2-44ec-9b30-0d9984433827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:48.895245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a66b8d-79d2-44ec-9b30-0d9984433827 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:48.895636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a66b8d-79d2-44ec-9b30-0d9984433827 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:48.895790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e64821c-8523-4c21-8b9c-29c6c25df6cf None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:48.896149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a66b8d-79d2-44ec-9b30-0d9984433827 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:48.897142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a66b8d-79d2-44ec-9b30-0d9984433827 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:48.897615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a66b8d-79d2-44ec-9b30-0d9984433827 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:48.899307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a66b8d-79d2-44ec-9b30-0d9984433827 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:48.905332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 403/808] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Jun 26 02:52:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:48.906253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a66b8d-79d2-44ec-9b30-0d9984433827 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-8e64821c-8523-4c21-8b9c-29c6c25df6cf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:48.911983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:48.913082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:48.914333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:48.914333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:48.914333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:48.914333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:48.914333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:48.914333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:48.914333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:48.914333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:48.914333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:52:48.914333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:48.916914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 406/809] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Jun 26 02:52:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:48.920393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64821c-8523-4c21-8b9c-29c6c25df6cf None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:48.930692 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:49.015208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a66b8d-79d2-44ec-9b30-0d9984433827 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:49.024064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e64821c-8523-4c21-8b9c-29c6c25df6cf None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:49.051170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dadef3-65cf-4be8-aef1-457297c02172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:49.051832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dadef3-65cf-4be8-aef1-457297c02172 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.052111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dadef3-65cf-4be8-aef1-457297c02172 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.052386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dadef3-65cf-4be8-aef1-457297c02172 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.053206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dadef3-65cf-4be8-aef1-457297c02172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:49.053509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dadef3-65cf-4be8-aef1-457297c02172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:49.055046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dadef3-65cf-4be8-aef1-457297c02172 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:49.060246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449b4e0-e298-48a1-b0a0-580e77fb1709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:49.060911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449b4e0-e298-48a1-b0a0-580e77fb1709 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.061196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449b4e0-e298-48a1-b0a0-580e77fb1709 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.061474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449b4e0-e298-48a1-b0a0-580e77fb1709 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.062216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449b4e0-e298-48a1-b0a0-580e77fb1709 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:49.062537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449b4e0-e298-48a1-b0a0-580e77fb1709 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:49.063880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449b4e0-e298-48a1-b0a0-580e77fb1709 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:49.072318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 404/810] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:49.072318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dadef3-65cf-4be8-aef1-457297c02172 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:49.073486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 407/811] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Jun 26 02:52:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:49.073486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69e4a2-ce49-4cb9-ab6b-7bf6d908c2c0 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.073486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69e4a2-ce49-4cb9-ab6b-7bf6d908c2c0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.073486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69e4a2-ce49-4cb9-ab6b-7bf6d908c2c0 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.073486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 408/812] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:49.074456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69e4a2-ce49-4cb9-ab6b-7bf6d908c2c0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:49.126327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:49.155339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:49.182908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69e4a2-ce49-4cb9-ab6b-7bf6d908c2c0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:49.184726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dadef3-65cf-4be8-aef1-457297c02172 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:49.219021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a20342-d43d-4594-bbfb-aec82959f520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:49.220089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a20342-d43d-4594-bbfb-aec82959f520 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.220373 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d08e8-23f4-4341-b526-89b9b5c8b0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:49.220710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a20342-d43d-4594-bbfb-aec82959f520 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.221037 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d08e8-23f4-4341-b526-89b9b5c8b0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.221285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a20342-d43d-4594-bbfb-aec82959f520 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.221392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d08e8-23f4-4341-b526-89b9b5c8b0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.221605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5d08e8-23f4-4341-b526-89b9b5c8b0a2 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.222458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d08e8-23f4-4341-b526-89b9b5c8b0a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:49.222767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d08e8-23f4-4341-b526-89b9b5c8b0a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:49.224172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5d08e8-23f4-4341-b526-89b9b5c8b0a2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:49.229649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a20342-d43d-4594-bbfb-aec82959f520 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:49.231253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a20342-d43d-4594-bbfb-aec82959f520 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:49.241136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 405/813] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Jun 26 02:52:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:49.241906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d08e8-23f4-4341-b526-89b9b5c8b0a2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e0a20342-d43d-4594-bbfb-aec82959f520 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:49.246611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:49.247706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:49.249124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:49.249124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:49.249124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:49.249124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:49.249124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:49.249124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:49.249124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:49.249124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:49.249124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:52:49.249124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:49.251308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 409/814] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Jun 26 02:52:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:49.252625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f742578-475c-4d01-b7e5-ffc92cc759e3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.253077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f742578-475c-4d01-b7e5-ffc92cc759e3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.253498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f742578-475c-4d01-b7e5-ffc92cc759e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.271167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9f742578-475c-4d01-b7e5-ffc92cc759e3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:49.272628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9f742578-475c-4d01-b7e5-ffc92cc759e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:49.342799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 410/815] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:49.343935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f742578-475c-4d01-b7e5-ffc92cc759e3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:49.357468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5d08e8-23f4-4341-b526-89b9b5c8b0a2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:49.365235 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:49.393364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209dfb4-e2dc-414a-aced-cd31c72f7c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:49.393953 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209dfb4-e2dc-414a-aced-cd31c72f7c13 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.394264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209dfb4-e2dc-414a-aced-cd31c72f7c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.394543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209dfb4-e2dc-414a-aced-cd31c72f7c13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.395278 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209dfb4-e2dc-414a-aced-cd31c72f7c13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:49.396648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c209dfb4-e2dc-414a-aced-cd31c72f7c13 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:49.420175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:49.430413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 406/816] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Jun 26 02:52:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:49.431250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209dfb4-e2dc-414a-aced-cd31c72f7c13 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:49.465912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f742578-475c-4d01-b7e5-ffc92cc759e3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:49.523292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b06c5c-6e16-414f-8798-19da2d0e11fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:49.524354 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06c5c-6e16-414f-8798-19da2d0e11fd None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.524881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06c5c-6e16-414f-8798-19da2d0e11fd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.525366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b06c5c-6e16-414f-8798-19da2d0e11fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.566160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209dfb4-e2dc-414a-aced-cd31c72f7c13 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:49.618249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffb505-6289-486b-8534-d451dff5ff40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:49.619143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffb505-6289-486b-8534-d451dff5ff40 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.620101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffb505-6289-486b-8534-d451dff5ff40 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.620384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeffb505-6289-486b-8534-d451dff5ff40 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.621230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeffb505-6289-486b-8534-d451dff5ff40 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:49.621631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeffb505-6289-486b-8534-d451dff5ff40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:49.623002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeffb505-6289-486b-8534-d451dff5ff40 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:49.636817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b06c5c-6e16-414f-8798-19da2d0e11fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:49.646790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b06c5c-6e16-414f-8798-19da2d0e11fd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:49.653426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 407/817] 10.222.0.13 () {56 vars in 1149 bytes} [Sun Jun 26 02:52:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:49.654817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffb505-6289-486b-8534-d451dff5ff40 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:49.686527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:49.721420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:49.771347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 411/818] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:52:49.772214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b06c5c-6e16-414f-8798-19da2d0e11fd None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:49.799087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeffb505-6289-486b-8534-d451dff5ff40 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:49.838069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c356e-5c02-40e3-aaa4-be2c931c69de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:49.838935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c356e-5c02-40e3-aaa4-be2c931c69de None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.839218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c356e-5c02-40e3-aaa4-be2c931c69de None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.839488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c356e-5c02-40e3-aaa4-be2c931c69de None admin] PATH_INFO: `/v3/services/network` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.840143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c356e-5c02-40e3-aaa4-be2c931c69de None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:49.841552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c356e-5c02-40e3-aaa4-be2c931c69de None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-056c356e-5c02-40e3-aaa4-be2c931c69de None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:49.853555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:49.854574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:49.855407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 408/819] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:49 2022] GET /identity/v3/services/network => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:49.856200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c356e-5c02-40e3-aaa4-be2c931c69de None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:49.917750 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b06c5c-6e16-414f-8798-19da2d0e11fd None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:49.960399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:49.964650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aa715-4369-44d2-9f57-3eeaafb44f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:49.965460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aa715-4369-44d2-9f57-3eeaafb44f43 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:49.965778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aa715-4369-44d2-9f57-3eeaafb44f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:49.966077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136aa715-4369-44d2-9f57-3eeaafb44f43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:49.977361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136aa715-4369-44d2-9f57-3eeaafb44f43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:49.978915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136aa715-4369-44d2-9f57-3eeaafb44f43 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:49.982231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c356e-5c02-40e3-aaa4-be2c931c69de None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-136aa715-4369-44d2-9f57-3eeaafb44f43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:49.997200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:49.998237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:49.999366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:49.999366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:49.999366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:49.999366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:50.003369 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 412/820] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:50.004252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aa715-4369-44d2-9f57-3eeaafb44f43 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:50.019682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:50.031324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4362-354c-4f79-ba55-3c214d175948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:50.033013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4362-354c-4f79-ba55-3c214d175948 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.033504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4362-354c-4f79-ba55-3c214d175948 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.034205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f4362-354c-4f79-ba55-3c214d175948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.048839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f4362-354c-4f79-ba55-3c214d175948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:50.051307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94f4362-354c-4f79-ba55-3c214d175948 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f94f4362-354c-4f79-ba55-3c214d175948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:50.073937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:50.075014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:50.076101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:50.076101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:50.076101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:50.076101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:50.078573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 409/821] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:50.079825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4362-354c-4f79-ba55-3c214d175948 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:50.116843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136aa715-4369-44d2-9f57-3eeaafb44f43 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:50.154327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a792741-d3ca-4dc2-bbf8-b9ec1b1769a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:50.155228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a792741-d3ca-4dc2-bbf8-b9ec1b1769a9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.155613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a792741-d3ca-4dc2-bbf8-b9ec1b1769a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.156062 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a792741-d3ca-4dc2-bbf8-b9ec1b1769a9 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.156966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a792741-d3ca-4dc2-bbf8-b9ec1b1769a9 None admin] RBAC: Adding query filter params (name=network) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:50.157375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a792741-d3ca-4dc2-bbf8-b9ec1b1769a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:50.158906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a792741-d3ca-4dc2-bbf8-b9ec1b1769a9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:50.164373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 413/822] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:52:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:50.165363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a792741-d3ca-4dc2-bbf8-b9ec1b1769a9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:50.202024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f4362-354c-4f79-ba55-3c214d175948 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:50.236678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d34e0-bf6e-45cf-ac2a-b15adb0db6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:50.237343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d34e0-bf6e-45cf-ac2a-b15adb0db6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.237620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d34e0-bf6e-45cf-ac2a-b15adb0db6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.237889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796d34e0-bf6e-45cf-ac2a-b15adb0db6e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.238634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d34e0-bf6e-45cf-ac2a-b15adb0db6e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:50.238938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d34e0-bf6e-45cf-ac2a-b15adb0db6e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:50.240307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796d34e0-bf6e-45cf-ac2a-b15adb0db6e5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:50.242725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:50.254232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:50.260025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 410/823] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:50.260025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d34e0-bf6e-45cf-ac2a-b15adb0db6e5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:50.286165 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a792741-d3ca-4dc2-bbf8-b9ec1b1769a9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:50.326453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7651f4-1f19-465c-bd41-c33ab5502185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:50.327435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7651f4-1f19-465c-bd41-c33ab5502185 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.327871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7651f4-1f19-465c-bd41-c33ab5502185 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.328297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7651f4-1f19-465c-bd41-c33ab5502185 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.329189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7651f4-1f19-465c-bd41-c33ab5502185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:50.329647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7651f4-1f19-465c-bd41-c33ab5502185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:50.331259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7651f4-1f19-465c-bd41-c33ab5502185 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:50.345601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 414/824] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:50.346812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7651f4-1f19-465c-bd41-c33ab5502185 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:50.461858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7651f4-1f19-465c-bd41-c33ab5502185 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:50.476513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796d34e0-bf6e-45cf-ac2a-b15adb0db6e5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:50.500519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a811c54-657a-466e-bcf9-d90b5305c3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:50.501578 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a811c54-657a-466e-bcf9-d90b5305c3e6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.502021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a811c54-657a-466e-bcf9-d90b5305c3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.502553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a811c54-657a-466e-bcf9-d90b5305c3e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.505716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:50.511727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a811c54-657a-466e-bcf9-d90b5305c3e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:50.511896 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cdd19-f78e-42fa-b268-b89116e05afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:50.512581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cdd19-f78e-42fa-b268-b89116e05afb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.512860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cdd19-f78e-42fa-b268-b89116e05afb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.513137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0cdd19-f78e-42fa-b268-b89116e05afb None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.513839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cdd19-f78e-42fa-b268-b89116e05afb None admin] RBAC: Adding query filter params (type=network) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:50.514172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cdd19-f78e-42fa-b268-b89116e05afb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:50.514348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a811c54-657a-466e-bcf9-d90b5305c3e6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:50.515511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0cdd19-f78e-42fa-b268-b89116e05afb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:50.526379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 411/825] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:52:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:50.527222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cdd19-f78e-42fa-b268-b89116e05afb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-6a811c54-657a-466e-bcf9-d90b5305c3e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:50.530547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:50.531579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:50.532606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:50.532606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:50.532606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:50.532606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:50.532606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:50.532606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:50.532606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:50.532606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:50.539198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 415/826] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:50.540318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a811c54-657a-466e-bcf9-d90b5305c3e6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:50.588568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:50.634890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0cdd19-f78e-42fa-b268-b89116e05afb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:50.653408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a811c54-657a-466e-bcf9-d90b5305c3e6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:50.671628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935a353-c07a-4869-98e3-e279203ce6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:50.672424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935a353-c07a-4869-98e3-e279203ce6ef None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.672708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935a353-c07a-4869-98e3-e279203ce6ef None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.672981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8935a353-c07a-4869-98e3-e279203ce6ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.679829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935a353-c07a-4869-98e3-e279203ce6ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:50.681267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8935a353-c07a-4869-98e3-e279203ce6ef None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:50.689588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119983ec-3452-4bec-8ce2-b28aac6e2dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:50.690268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119983ec-3452-4bec-8ce2-b28aac6e2dae None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.690603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119983ec-3452-4bec-8ce2-b28aac6e2dae None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.690882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119983ec-3452-4bec-8ce2-b28aac6e2dae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.691613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119983ec-3452-4bec-8ce2-b28aac6e2dae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:50.692964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119983ec-3452-4bec-8ce2-b28aac6e2dae None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8935a353-c07a-4869-98e3-e279203ce6ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:50.695678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:50.696525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:50.697395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:50.697395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:50.697395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:50.697395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:50.697395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:50.697395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:50.697395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:50.697395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:50.697395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 412/827] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:50.699069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935a353-c07a-4869-98e3-e279203ce6ef None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:50.720914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 416/828] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Jun 26 02:52:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:50.722544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95ad06-fdeb-4180-99aa-bf1eb1abc0fe None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.723108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95ad06-fdeb-4180-99aa-bf1eb1abc0fe None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.723108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae95ad06-fdeb-4180-99aa-bf1eb1abc0fe None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.724559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 417/829] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:50.729625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95ad06-fdeb-4180-99aa-bf1eb1abc0fe None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:50.812553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:50.845809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8935a353-c07a-4869-98e3-e279203ce6ef None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:50.873148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:50.888941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae95ad06-fdeb-4180-99aa-bf1eb1abc0fe None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:50.890564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c127bc-50e3-4c33-a895-22680f43f423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:50.891243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c127bc-50e3-4c33-a895-22680f43f423 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.891528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c127bc-50e3-4c33-a895-22680f43f423 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.891808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c127bc-50e3-4c33-a895-22680f43f423 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.892604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c127bc-50e3-4c33-a895-22680f43f423 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:50.892921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c127bc-50e3-4c33-a895-22680f43f423 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:50.894384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c127bc-50e3-4c33-a895-22680f43f423 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:50.908151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 413/830] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:50.909554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729385e-94d7-420c-be24-48abc7eba3f5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.909852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729385e-94d7-420c-be24-48abc7eba3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.910994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729385e-94d7-420c-be24-48abc7eba3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.931207 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c729385e-94d7-420c-be24-48abc7eba3f5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:50.933076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c729385e-94d7-420c-be24-48abc7eba3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:50.937182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ecf26-d781-4ac2-9aed-d574f01baa76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:50.938152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ecf26-d781-4ac2-9aed-d574f01baa76 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:50.938680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ecf26-d781-4ac2-9aed-d574f01baa76 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:50.939171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3ecf26-d781-4ac2-9aed-d574f01baa76 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:50.940208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ecf26-d781-4ac2-9aed-d574f01baa76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:50.940725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ecf26-d781-4ac2-9aed-d574f01baa76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:50.942839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3ecf26-d781-4ac2-9aed-d574f01baa76 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:50.965070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 418/831] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:50.965070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ecf26-d781-4ac2-9aed-d574f01baa76 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:51.011221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 414/832] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:51.012424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729385e-94d7-420c-be24-48abc7eba3f5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:51.087595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3ecf26-d781-4ac2-9aed-d574f01baa76 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:51.098347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:51.114691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729385e-94d7-420c-be24-48abc7eba3f5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:51.126596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3a094-a9f1-4c50-a131-86a77c1f78fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:51.127113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:51.127477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3a094-a9f1-4c50-a131-86a77c1f78fa None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:51.127760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3a094-a9f1-4c50-a131-86a77c1f78fa None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:51.128048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3a094-a9f1-4c50-a131-86a77c1f78fa None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/828fd11b79d045dc85e0973cd255bcf7/roles/10b1c5eed2754da5adfd5217256adba8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:51.144183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3a094-a9f1-4c50-a131-86a77c1f78fa None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=828fd11b79d045dc85e0973cd255bcf7, role_id=10b1c5eed2754da5adfd5217256adba8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:51.145637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3a094-a9f1-4c50-a131-86a77c1f78fa None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:51.156109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1542b-e28f-4db0-8596-97111fd6bd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:51.156821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1542b-e28f-4db0-8596-97111fd6bd98 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:51.157115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1542b-e28f-4db0-8596-97111fd6bd98 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:51.157372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1542b-e28f-4db0-8596-97111fd6bd98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:51.158251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1542b-e28f-4db0-8596-97111fd6bd98 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=9ee0150dfb3f42709029e42d0fd9105d) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:51.158562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1542b-e28f-4db0-8596-97111fd6bd98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:51.159928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 419/833] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:52:50 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/828fd11b79d045dc85e0973cd255bcf7/roles/10b1c5eed2754da5adfd5217256adba8 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:52:51.160598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3a094-a9f1-4c50-a131-86a77c1f78fa None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:51.166980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1542b-e28f-4db0-8596-97111fd6bd98 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:51.177138 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 415/834] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:51 2022] GET /identity/v3/role_assignments?user.id=9ee0150dfb3f42709029e42d0fd9105d&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:51.287396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3a094-a9f1-4c50-a131-86a77c1f78fa None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:51.331039 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd49ff5-1fed-42c9-b281-885cebd35d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:51.331933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd49ff5-1fed-42c9-b281-885cebd35d2a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:51.332227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd49ff5-1fed-42c9-b281-885cebd35d2a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:51.332509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd49ff5-1fed-42c9-b281-885cebd35d2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:51.346134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd49ff5-1fed-42c9-b281-885cebd35d2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-dcd49ff5-1fed-42c9-b281-885cebd35d2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:51.354762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:51.355753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:52:51.356585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:51.361223 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 420/835] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:51.362672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1542b-e28f-4db0-8596-97111fd6bd98 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:51.480109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1542b-e28f-4db0-8596-97111fd6bd98 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:51.514581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76328014-f4a4-4a80-9c9d-d487bc8e902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:51.515586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76328014-f4a4-4a80-9c9d-d487bc8e902e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:51.516048 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76328014-f4a4-4a80-9c9d-d487bc8e902e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:51.516504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76328014-f4a4-4a80-9c9d-d487bc8e902e None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:51.517434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76328014-f4a4-4a80-9c9d-d487bc8e902e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:51.517932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76328014-f4a4-4a80-9c9d-d487bc8e902e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:51.519533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76328014-f4a4-4a80-9c9d-d487bc8e902e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:51.529127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 416/836] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:52:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:51.534546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd49ff5-1fed-42c9-b281-885cebd35d2a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:51.549559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde704ca-3182-4e0d-ba27-8d0f89ed0de8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:51.549794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde704ca-3182-4e0d-ba27-8d0f89ed0de8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:51.550066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde704ca-3182-4e0d-ba27-8d0f89ed0de8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:51.551717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 417/837] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:51.556564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905610-a88e-4b99-9bca-5502752e3376 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:51.556894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905610-a88e-4b99-9bca-5502752e3376 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:51.557176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45905610-a88e-4b99-9bca-5502752e3376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:51.557679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:51.576555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-45905610-a88e-4b99-9bca-5502752e3376 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:51.578614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-45905610-a88e-4b99-9bca-5502752e3376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:51.614434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:51.660829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 418/838] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:52:51.665437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd49ff5-1fed-42c9-b281-885cebd35d2a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:51.688830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45905610-a88e-4b99-9bca-5502752e3376 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:51.702883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bd146-686a-4950-a109-e8f7861c6b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:51.704009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bd146-686a-4950-a109-e8f7861c6b56 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:51.704510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bd146-686a-4950-a109-e8f7861c6b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:51.704950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bd146-686a-4950-a109-e8f7861c6b56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:51.716551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bd146-686a-4950-a109-e8f7861c6b56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:51.718164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bd146-686a-4950-a109-e8f7861c6b56 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-8a3bd146-686a-4950-a109-e8f7861c6b56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:51.736025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:51.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:51.738749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:51.738749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:51.738749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:51.738749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:51.738749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 421/839] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:51.745155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bd146-686a-4950-a109-e8f7861c6b56 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:51.809606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45905610-a88e-4b99-9bca-5502752e3376 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:51.833162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:51.848498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd74df-fa14-4d5e-ac4a-edf43a7eadc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:51.849455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd74df-fa14-4d5e-ac4a-edf43a7eadc5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:51.849855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd74df-fa14-4d5e-ac4a-edf43a7eadc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:51.850257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cd74df-fa14-4d5e-ac4a-edf43a7eadc5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:51.851010 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd74df-fa14-4d5e-ac4a-edf43a7eadc5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:51.853019 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cd74df-fa14-4d5e-ac4a-edf43a7eadc5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f0cd74df-fa14-4d5e-ac4a-edf43a7eadc5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:51.866271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:51.867488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 26 02:52:51.868556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:51.870770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 419/840] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:51.876739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd74df-fa14-4d5e-ac4a-edf43a7eadc5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:51.877955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bd146-686a-4950-a109-e8f7861c6b56 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:51.914251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:51.916003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df511d-1aa4-472b-b994-93d474bbcd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:51.917028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df511d-1aa4-472b-b994-93d474bbcd49 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:51.917470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df511d-1aa4-472b-b994-93d474bbcd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:51.917768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96df511d-1aa4-472b-b994-93d474bbcd49 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:51.918521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df511d-1aa4-472b-b994-93d474bbcd49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:51.918831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df511d-1aa4-472b-b994-93d474bbcd49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:51.920522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96df511d-1aa4-472b-b994-93d474bbcd49 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:51.933623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 422/841] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:51.938995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df511d-1aa4-472b-b994-93d474bbcd49 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:51.989806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cd74df-fa14-4d5e-ac4a-edf43a7eadc5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:52.026804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee2f84-5b3c-4cd5-8f6f-ebcb8366f096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:52.027474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2f84-5b3c-4cd5-8f6f-ebcb8366f096 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:52.027848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2f84-5b3c-4cd5-8f6f-ebcb8366f096 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:52.028124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bee2f84-5b3c-4cd5-8f6f-ebcb8366f096 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:52.028832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee2f84-5b3c-4cd5-8f6f-ebcb8366f096 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:52.029140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee2f84-5b3c-4cd5-8f6f-ebcb8366f096 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:52.030639 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bee2f84-5b3c-4cd5-8f6f-ebcb8366f096 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:52.040335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 420/842] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:52.041357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee2f84-5b3c-4cd5-8f6f-ebcb8366f096 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:52.050290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96df511d-1aa4-472b-b994-93d474bbcd49 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:52.082527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:52.095273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60e181-2e72-4b09-a76e-116e6bbdfc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:52.096072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60e181-2e72-4b09-a76e-116e6bbdfc8f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:52.096415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60e181-2e72-4b09-a76e-116e6bbdfc8f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:52.096696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60e181-2e72-4b09-a76e-116e6bbdfc8f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:52.107344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60e181-2e72-4b09-a76e-116e6bbdfc8f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:52.109245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60e181-2e72-4b09-a76e-116e6bbdfc8f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0d60e181-2e72-4b09-a76e-116e6bbdfc8f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:52.127355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:52.128472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:52.129400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:52.129400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:52.129400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:52.129400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:52.129400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:52.129400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:52.129400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:52.129400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:52.129400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 26 02:52:52.129400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:52.131715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 423/843] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:52:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:52.142082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60e181-2e72-4b09-a76e-116e6bbdfc8f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:52.156457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:52.161606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bee2f84-5b3c-4cd5-8f6f-ebcb8366f096 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:52.196570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89122f0-2891-417f-885c-c078b28509fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:52.197322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89122f0-2891-417f-885c-c078b28509fc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:52.198238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89122f0-2891-417f-885c-c078b28509fc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:52.198585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89122f0-2891-417f-885c-c078b28509fc None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:52.199370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89122f0-2891-417f-885c-c078b28509fc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:52.199743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89122f0-2891-417f-885c-c078b28509fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:52.201179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89122f0-2891-417f-885c-c078b28509fc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:52.206220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 421/844] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:52.254563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60e181-2e72-4b09-a76e-116e6bbdfc8f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:52.292914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf250c-5f15-439f-8eab-b1ff81784601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:52.293601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf250c-5f15-439f-8eab-b1ff81784601 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:52.293887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf250c-5f15-439f-8eab-b1ff81784601 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:52.294196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcf250c-5f15-439f-8eab-b1ff81784601 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:52.295048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf250c-5f15-439f-8eab-b1ff81784601 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:52.295361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf250c-5f15-439f-8eab-b1ff81784601 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:52.296915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcf250c-5f15-439f-8eab-b1ff81784601 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:52.316915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 424/845] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Jun 26 02:52:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:52.323730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89122f0-2891-417f-885c-c078b28509fc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:52.431432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89122f0-2891-417f-885c-c078b28509fc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:52.470825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a1030-6510-4905-958b-ab6c32db92fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:52.471872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a1030-6510-4905-958b-ab6c32db92fb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:52.472373 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a1030-6510-4905-958b-ab6c32db92fb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:52.472887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a1030-6510-4905-958b-ab6c32db92fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:52.502788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:52.546516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a1030-6510-4905-958b-ab6c32db92fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:52.547958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a1030-6510-4905-958b-ab6c32db92fb None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:52.660263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 422/846] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:52:52.662148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf250c-5f15-439f-8eab-b1ff81784601 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:52.712694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:52.768185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcf250c-5f15-439f-8eab-b1ff81784601 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:52.803306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724dea9-02c0-442a-8b89-700719ab9b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:52.804108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724dea9-02c0-442a-8b89-700719ab9b93 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:52.804390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724dea9-02c0-442a-8b89-700719ab9b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:52.804664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724dea9-02c0-442a-8b89-700719ab9b93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:52.815186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724dea9-02c0-442a-8b89-700719ab9b93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:52.816642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724dea9-02c0-442a-8b89-700719ab9b93 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5724dea9-02c0-442a-8b89-700719ab9b93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:52.833954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:52.834876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:52.835741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:52.835741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:52.835741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:52.835741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:52.835741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 425/847] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:52.841477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a1030-6510-4905-958b-ab6c32db92fb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:52.910871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:52.957869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a1030-6510-4905-958b-ab6c32db92fb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:52.995014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6107d-e106-4f07-bfc3-969055ec71f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:52.996022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6107d-e106-4f07-bfc3-969055ec71f7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:52.996522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6107d-e106-4f07-bfc3-969055ec71f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:52.997030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6107d-e106-4f07-bfc3-969055ec71f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:52.997956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6107d-e106-4f07-bfc3-969055ec71f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:52.998501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6107d-e106-4f07-bfc3-969055ec71f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:53.000327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6107d-e106-4f07-bfc3-969055ec71f7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:53.017285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 423/848] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:53.019133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724dea9-02c0-442a-8b89-700719ab9b93 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:53.121293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:53.124399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724dea9-02c0-442a-8b89-700719ab9b93 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:53.159593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a59ed0-ca74-4993-b3b2-d8d8ade58d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:53.160392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a59ed0-ca74-4993-b3b2-d8d8ade58d72 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:53.160672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a59ed0-ca74-4993-b3b2-d8d8ade58d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:53.160946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a59ed0-ca74-4993-b3b2-d8d8ade58d72 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:53.167799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a59ed0-ca74-4993-b3b2-d8d8ade58d72 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:53.169229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a59ed0-ca74-4993-b3b2-d8d8ade58d72 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-59a59ed0-ca74-4993-b3b2-d8d8ade58d72 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:53.182763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:53.183628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:53.184694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:53.184694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:53.184694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:53.184694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:53.184694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:53.184694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:53.184694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:53.184694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:53.184694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 426/849] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:53.191396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6107d-e106-4f07-bfc3-969055ec71f7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:53.300615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6107d-e106-4f07-bfc3-969055ec71f7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:53.319188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:53.339067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b13764-82b4-4561-a72c-8bdb44a77482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:53.340080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b13764-82b4-4561-a72c-8bdb44a77482 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:53.340591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b13764-82b4-4561-a72c-8bdb44a77482 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:53.341088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b13764-82b4-4561-a72c-8bdb44a77482 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:53.342102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b13764-82b4-4561-a72c-8bdb44a77482 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:53.342621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b13764-82b4-4561-a72c-8bdb44a77482 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:53.344242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b13764-82b4-4561-a72c-8bdb44a77482 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:53.362142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 424/850] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:53.364307 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a59ed0-ca74-4993-b3b2-d8d8ade58d72 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:53.380481 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1b572-35f5-4e0b-9ffa-051d233e9e73 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:53.381091 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1b572-35f5-4e0b-9ffa-051d233e9e73 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:53.381573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db1b572-35f5-4e0b-9ffa-051d233e9e73 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:53.383543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 425/851] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:53.390370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50af9c-4693-4a43-877f-819c5fee5f55 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:53.390905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50af9c-4693-4a43-877f-819c5fee5f55 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:53.391392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be50af9c-4693-4a43-877f-819c5fee5f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:53.414163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-be50af9c-4693-4a43-877f-819c5fee5f55 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:53.414163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-be50af9c-4693-4a43-877f-819c5fee5f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:53.479966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a59ed0-ca74-4993-b3b2-d8d8ade58d72 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:53.481813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 426/852] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:52:53.483000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f414aa-b738-4302-af8f-dda30191d964 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:53.483287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f414aa-b738-4302-af8f-dda30191d964 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:53.483562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f414aa-b738-4302-af8f-dda30191d964 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:53.488792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 427/853] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:53.491425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c95a2-84c9-49f2-906f-94203d58f39e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:53.491708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c95a2-84c9-49f2-906f-94203d58f39e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:53.491996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c95a2-84c9-49f2-906f-94203d58f39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:53.511303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0e3c95a2-84c9-49f2-906f-94203d58f39e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:53.512485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0e3c95a2-84c9-49f2-906f-94203d58f39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:53.520428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a474efb-44d7-4fc3-be53-8968a882bea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:53.521114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a474efb-44d7-4fc3-be53-8968a882bea3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:53.521404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a474efb-44d7-4fc3-be53-8968a882bea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:53.521686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a474efb-44d7-4fc3-be53-8968a882bea3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:53.522559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a474efb-44d7-4fc3-be53-8968a882bea3 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=60f006c99809427496b026c17a663995) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:53.522876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a474efb-44d7-4fc3-be53-8968a882bea3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:53.524358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a474efb-44d7-4fc3-be53-8968a882bea3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:53.542095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 427/854] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:53 2022] GET /identity/v3/role_assignments?user.id=60f006c99809427496b026c17a663995&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:53.542095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a474efb-44d7-4fc3-be53-8968a882bea3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:53.574686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:53.614429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 428/855] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:53.631721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c95a2-84c9-49f2-906f-94203d58f39e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:53.665551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a474efb-44d7-4fc3-be53-8968a882bea3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:53.673502 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:53.706005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b211808-e722-404b-b061-5777aa3604ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:53.706858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b211808-e722-404b-b061-5777aa3604ac None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:53.707149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b211808-e722-404b-b061-5777aa3604ac None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:53.707428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b211808-e722-404b-b061-5777aa3604ac None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:53.708209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b211808-e722-404b-b061-5777aa3604ac None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:53.709582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b211808-e722-404b-b061-5777aa3604ac None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-5b211808-e722-404b-b061-5777aa3604ac None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:53.723333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:53.724258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:53.725080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 428/856] 10.222.0.13 () {56 vars in 1132 bytes} [Sun Jun 26 02:52:53 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:53.732374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b211808-e722-404b-b061-5777aa3604ac None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:53.744859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c95a2-84c9-49f2-906f-94203d58f39e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:53.786775 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374004f-1d26-4395-aa2a-3c4b608745f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:53.787872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374004f-1d26-4395-aa2a-3c4b608745f2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:53.788156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374004f-1d26-4395-aa2a-3c4b608745f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:53.788582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9374004f-1d26-4395-aa2a-3c4b608745f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:53.806618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9374004f-1d26-4395-aa2a-3c4b608745f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9374004f-1d26-4395-aa2a-3c4b608745f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:53.814372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:53.815323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:53.816567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 429/857] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:52:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:53.821546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374004f-1d26-4395-aa2a-3c4b608745f2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:53.855063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:53.868757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b211808-e722-404b-b061-5777aa3604ac None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:53.909237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03340c-8bdb-42cd-a71d-1efa044fe99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:53.909896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03340c-8bdb-42cd-a71d-1efa044fe99a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:53.910206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03340c-8bdb-42cd-a71d-1efa044fe99a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:53.910477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03340c-8bdb-42cd-a71d-1efa044fe99a None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:53.911185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03340c-8bdb-42cd-a71d-1efa044fe99a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:53.911489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03340c-8bdb-42cd-a71d-1efa044fe99a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:53.912847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e03340c-8bdb-42cd-a71d-1efa044fe99a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:53.918258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 429/858] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:52:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:53.925805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03340c-8bdb-42cd-a71d-1efa044fe99a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:53.957862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:53.962312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9374004f-1d26-4395-aa2a-3c4b608745f2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:53.999673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04faa85-e8ce-4bb0-ba5a-b779ea4c6fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:54.000416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04faa85-e8ce-4bb0-ba5a-b779ea4c6fea None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:54.000619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04faa85-e8ce-4bb0-ba5a-b779ea4c6fea None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:54.000886 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04faa85-e8ce-4bb0-ba5a-b779ea4c6fea None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:54.001625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04faa85-e8ce-4bb0-ba5a-b779ea4c6fea None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:54.001930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04faa85-e8ce-4bb0-ba5a-b779ea4c6fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:54.003325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04faa85-e8ce-4bb0-ba5a-b779ea4c6fea None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:54.017115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 430/859] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:52:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:54.018251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04faa85-e8ce-4bb0-ba5a-b779ea4c6fea None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:54.040754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03340c-8bdb-42cd-a71d-1efa044fe99a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:54.080978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c26d1-41e4-40ba-8fd7-f8d55dca863d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:54.081667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c26d1-41e4-40ba-8fd7-f8d55dca863d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:54.081952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c26d1-41e4-40ba-8fd7-f8d55dca863d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:54.082335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c26d1-41e4-40ba-8fd7-f8d55dca863d None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:54.083054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c26d1-41e4-40ba-8fd7-f8d55dca863d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:54.083366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c26d1-41e4-40ba-8fd7-f8d55dca863d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:54.084750 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1c26d1-41e4-40ba-8fd7-f8d55dca863d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:54.098274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 430/860] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:52:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:54.141412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04faa85-e8ce-4bb0-ba5a-b779ea4c6fea None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:54.181189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b6d145-59e7-4b91-8c23-d5a01c55ec52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:54.182588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6d145-59e7-4b91-8c23-d5a01c55ec52 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:54.182870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6d145-59e7-4b91-8c23-d5a01c55ec52 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:54.183142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b6d145-59e7-4b91-8c23-d5a01c55ec52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:54.195313 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6d145-59e7-4b91-8c23-d5a01c55ec52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:54.196771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b6d145-59e7-4b91-8c23-d5a01c55ec52 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-d8b6d145-59e7-4b91-8c23-d5a01c55ec52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:54.216044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:54.217035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:54.218409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:54.218409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:54.218409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:54.218409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:54.218409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 431/861] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:54.223260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c26d1-41e4-40ba-8fd7-f8d55dca863d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:54.360337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:54.431331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07101e-8b95-4b92-a586-4d0d9af4a717 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:54.431529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07101e-8b95-4b92-a586-4d0d9af4a717 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:54.432325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07101e-8b95-4b92-a586-4d0d9af4a717 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:54.434023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 432/862] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:54.440203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd3dd-182a-4644-b7d0-191c46a69b90 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:54.440648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd3dd-182a-4644-b7d0-191c46a69b90 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:54.440930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131dd3dd-182a-4644-b7d0-191c46a69b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:54.453694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c26d1-41e4-40ba-8fd7-f8d55dca863d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:54.458268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-131dd3dd-182a-4644-b7d0-191c46a69b90 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:54.459458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-131dd3dd-182a-4644-b7d0-191c46a69b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:54.467555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:54.493017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b82ca-5817-4de2-ae56-eca7bae7bac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:54.493695 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b82ca-5817-4de2-ae56-eca7bae7bac5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:54.493982 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b82ca-5817-4de2-ae56-eca7bae7bac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:54.494816 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b82ca-5817-4de2-ae56-eca7bae7bac5 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:54.495682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b82ca-5817-4de2-ae56-eca7bae7bac5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:54.495996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b82ca-5817-4de2-ae56-eca7bae7bac5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:54.497406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7b82ca-5817-4de2-ae56-eca7bae7bac5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:54.509813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 431/863] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:54.516545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b82ca-5817-4de2-ae56-eca7bae7bac5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:54.541030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 433/864] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:54.565774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dd3dd-182a-4644-b7d0-191c46a69b90 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:54.610585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:54.644938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b82ca-5817-4de2-ae56-eca7bae7bac5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:54.680781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19db19-8e25-4840-9324-829fa9ea2484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:54.682116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19db19-8e25-4840-9324-829fa9ea2484 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:54.682607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19db19-8e25-4840-9324-829fa9ea2484 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:54.683101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19db19-8e25-4840-9324-829fa9ea2484 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:54.691167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19db19-8e25-4840-9324-829fa9ea2484 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:54.692908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19db19-8e25-4840-9324-829fa9ea2484 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:54.704028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131dd3dd-182a-4644-b7d0-191c46a69b90 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ab19db19-8e25-4840-9324-829fa9ea2484 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:54.709276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:54.710249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:54.711224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:54.711224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:54.711224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:54.711224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:54.711224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:54.711224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:54.711224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:54.711224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:54.711224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:52:54.711224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:54.719031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 432/865] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Jun 26 02:52:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:54.720391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19db19-8e25-4840-9324-829fa9ea2484 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:54.751903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a874f-3f81-4251-938d-1940e995dafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:54.752725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a874f-3f81-4251-938d-1940e995dafa None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:54.753109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a874f-3f81-4251-938d-1940e995dafa None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:54.753388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a874f-3f81-4251-938d-1940e995dafa None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:54.754157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a874f-3f81-4251-938d-1940e995dafa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:54.756535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a874f-3f81-4251-938d-1940e995dafa None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:54.782264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 434/866] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:52:54 2022] POST /identity/v3/services => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:52:54.846279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19db19-8e25-4840-9324-829fa9ea2484 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:54.900140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:54.907986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab2bf3-daaf-4acf-9111-f138a39a99d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:54.908674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab2bf3-daaf-4acf-9111-f138a39a99d3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:54.908956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab2bf3-daaf-4acf-9111-f138a39a99d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:54.909229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ab2bf3-daaf-4acf-9111-f138a39a99d3 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:54.910097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab2bf3-daaf-4acf-9111-f138a39a99d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:54.910415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab2bf3-daaf-4acf-9111-f138a39a99d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:54.911964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ab2bf3-daaf-4acf-9111-f138a39a99d3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:54.933319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 433/867] 10.222.0.13 () {56 vars in 1149 bytes} [Sun Jun 26 02:52:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:54.939207 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a874f-3f81-4251-938d-1940e995dafa None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:54.942376 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:55.046984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a874f-3f81-4251-938d-1940e995dafa None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:55.081484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e1191-11ab-4239-874b-797c2d5947b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:55.082176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e1191-11ab-4239-874b-797c2d5947b2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:55.082462 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e1191-11ab-4239-874b-797c2d5947b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:55.082738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205e1191-11ab-4239-874b-797c2d5947b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:55.152043 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e1191-11ab-4239-874b-797c2d5947b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:55.153489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205e1191-11ab-4239-874b-797c2d5947b2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:55.155620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:55.258099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 435/868] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:54 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:52:55.264550 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab2bf3-daaf-4acf-9111-f138a39a99d3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:55.370739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ab2bf3-daaf-4acf-9111-f138a39a99d3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:55.405704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6268a4-ef08-40ea-a8b0-4b99c6ed9aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:55.406886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6268a4-ef08-40ea-a8b0-4b99c6ed9aaf None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:55.407390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6268a4-ef08-40ea-a8b0-4b99c6ed9aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:55.407878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6268a4-ef08-40ea-a8b0-4b99c6ed9aaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:55.419379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6268a4-ef08-40ea-a8b0-4b99c6ed9aaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:55.421076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6268a4-ef08-40ea-a8b0-4b99c6ed9aaf None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-cb6268a4-ef08-40ea-a8b0-4b99c6ed9aaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:55.439162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:55.440048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:55.440914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:55.440914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:55.440914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:55.440914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:55.443075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 434/869] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:55.448376 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e1191-11ab-4239-874b-797c2d5947b2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:55.549802 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205e1191-11ab-4239-874b-797c2d5947b2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:55.551301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:55.585143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839b490e-f7a1-4162-bfc5-e0815196e881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:55.585811 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b490e-f7a1-4162-bfc5-e0815196e881 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:55.586118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b490e-f7a1-4162-bfc5-e0815196e881 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:55.586397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839b490e-f7a1-4162-bfc5-e0815196e881 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:55.587117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b490e-f7a1-4162-bfc5-e0815196e881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:55.587424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b490e-f7a1-4162-bfc5-e0815196e881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:55.588810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839b490e-f7a1-4162-bfc5-e0815196e881 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:55.601582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 436/870] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:55.611161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6268a4-ef08-40ea-a8b0-4b99c6ed9aaf None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:55.654858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb305ab-e150-4bd1-8467-5a7be2875f46 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:55.655211 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb305ab-e150-4bd1-8467-5a7be2875f46 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:55.655421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb305ab-e150-4bd1-8467-5a7be2875f46 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:55.657084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 437/871] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:55.662486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:55.663631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e95df-08c7-4cae-a07f-6da0ceb93351 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:55.664248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e95df-08c7-4cae-a07f-6da0ceb93351 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:55.666196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527e95df-08c7-4cae-a07f-6da0ceb93351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:55.684032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-527e95df-08c7-4cae-a07f-6da0ceb93351 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:55.685421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-527e95df-08c7-4cae-a07f-6da0ceb93351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:55.724170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6268a4-ef08-40ea-a8b0-4b99c6ed9aaf None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:55.759643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 438/872] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:55 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:55.761664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df851-7173-4a4e-bd56-edb4a18e980e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:55.766084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df851-7173-4a4e-bd56-edb4a18e980e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:55.766084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df851-7173-4a4e-bd56-edb4a18e980e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:55.766084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544df851-7173-4a4e-bd56-edb4a18e980e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:55.772698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544df851-7173-4a4e-bd56-edb4a18e980e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:55.774391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544df851-7173-4a4e-bd56-edb4a18e980e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:55.781930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e95df-08c7-4cae-a07f-6da0ceb93351 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-544df851-7173-4a4e-bd56-edb4a18e980e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:52:55.789040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:55.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:55.790803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:52:55.790803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:52:55.790803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:52:55.790803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:52:55.790803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:52:55.790803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:52:55.790803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:52:55.790803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:55.790803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 435/873] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:52:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:55.797806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df851-7173-4a4e-bd56-edb4a18e980e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:55.896057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:55.904929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527e95df-08c7-4cae-a07f-6da0ceb93351 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:55.917620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544df851-7173-4a4e-bd56-edb4a18e980e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:55.919789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:55.942311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4529221-cf6f-466b-a03e-1ff954061f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:55.943124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4529221-cf6f-466b-a03e-1ff954061f91 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:55.943504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4529221-cf6f-466b-a03e-1ff954061f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:55.943676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4529221-cf6f-466b-a03e-1ff954061f91 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:55.944333 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4529221-cf6f-466b-a03e-1ff954061f91 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:55.945816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4529221-cf6f-466b-a03e-1ff954061f91 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:55.958327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19863e4-1aa8-4230-8709-ae4f49050a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:55.959293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19863e4-1aa8-4230-8709-ae4f49050a8a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-f4529221-cf6f-466b-a03e-1ff954061f91 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:55.959412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:55.960536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:55.961570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 439/874] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:52:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:55.967654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19863e4-1aa8-4230-8709-ae4f49050a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:55.967654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19863e4-1aa8-4230-8709-ae4f49050a8a None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:55.967654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19863e4-1aa8-4230-8709-ae4f49050a8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:55.968167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19863e4-1aa8-4230-8709-ae4f49050a8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:55.969345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19863e4-1aa8-4230-8709-ae4f49050a8a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:55.971003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4529221-cf6f-466b-a03e-1ff954061f91 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:55.986126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 436/875] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:52:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:55.989904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19863e4-1aa8-4230-8709-ae4f49050a8a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:56.088779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4529221-cf6f-466b-a03e-1ff954061f91 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:56.099578 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19863e4-1aa8-4230-8709-ae4f49050a8a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:56.125054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3b40e-c327-4070-8b51-565f52640318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:56.125995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b40e-c327-4070-8b51-565f52640318 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:56.126421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b40e-c327-4070-8b51-565f52640318 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:56.126797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee3b40e-c327-4070-8b51-565f52640318 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:56.127606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3b40e-c327-4070-8b51-565f52640318 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:56.128017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3b40e-c327-4070-8b51-565f52640318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:56.129521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee3b40e-c327-4070-8b51-565f52640318 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:56.138954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 440/876] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Jun 26 02:52:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:56.140368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3b40e-c327-4070-8b51-565f52640318 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:56.141527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631c644-c287-47be-9d79-60bf4a13bd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:56.142229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631c644-c287-47be-9d79-60bf4a13bd60 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:56.142515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631c644-c287-47be-9d79-60bf4a13bd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:56.142790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631c644-c287-47be-9d79-60bf4a13bd60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:56.143631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631c644-c287-47be-9d79-60bf4a13bd60 None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=828fd11b79d045dc85e0973cd255bcf7) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:56.143948 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631c644-c287-47be-9d79-60bf4a13bd60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:56.146592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9631c644-c287-47be-9d79-60bf4a13bd60 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:56.153586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 437/877] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:52:55 2022] GET /identity/v3/role_assignments?user.id=828fd11b79d045dc85e0973cd255bcf7&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:56.198431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:56.200054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e22b5-c68f-49fb-b236-2100c8a832ee None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:56.200362 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e22b5-c68f-49fb-b236-2100c8a832ee None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:56.200664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077e22b5-c68f-49fb-b236-2100c8a832ee None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:56.203005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 438/878] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:56.208689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1574eb-87c1-45dc-a5e7-e5255deecf74 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:56.208999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1574eb-87c1-45dc-a5e7-e5255deecf74 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:56.209255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1574eb-87c1-45dc-a5e7-e5255deecf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:56.229262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8f1574eb-87c1-45dc-a5e7-e5255deecf74 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:56.230664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8f1574eb-87c1-45dc-a5e7-e5255deecf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:56.255914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:56.271444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee3b40e-c327-4070-8b51-565f52640318 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:56.308996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbfc13-45f9-4ff8-b456-d4a23f2eb54c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:56.310069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbfc13-45f9-4ff8-b456-d4a23f2eb54c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:56.310359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbfc13-45f9-4ff8-b456-d4a23f2eb54c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:56.310634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbfc13-45f9-4ff8-b456-d4a23f2eb54c None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:56.311340 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbfc13-45f9-4ff8-b456-d4a23f2eb54c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:56.311645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbfc13-45f9-4ff8-b456-d4a23f2eb54c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:56.312344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 439/879] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:56 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:52:56.313077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbfc13-45f9-4ff8-b456-d4a23f2eb54c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:56.318590 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 441/880] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Jun 26 02:52:56 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:56.336083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1574eb-87c1-45dc-a5e7-e5255deecf74 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:56.448358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1574eb-87c1-45dc-a5e7-e5255deecf74 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:56.486643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a348c-ddff-4671-9ada-c3fc53f78d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:56.487381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a348c-ddff-4671-9ada-c3fc53f78d68 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:56.487665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a348c-ddff-4671-9ada-c3fc53f78d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:56.488029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a348c-ddff-4671-9ada-c3fc53f78d68 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:56.489126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a348c-ddff-4671-9ada-c3fc53f78d68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:56.491068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a348c-ddff-4671-9ada-c3fc53f78d68 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:56.507574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 440/881] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:52:56 2022] POST /identity/v3/services => generated 269 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:57.012141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847fa26-3cc8-4de0-b129-d1ba0cba5363 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:57.013138 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847fa26-3cc8-4de0-b129-d1ba0cba5363 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:57.013648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847fa26-3cc8-4de0-b129-d1ba0cba5363 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:57.015530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 442/882] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:57.021078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616605a9-0a96-4f8c-bc53-ec9dfba90f14 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:57.021911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616605a9-0a96-4f8c-bc53-ec9dfba90f14 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:57.022416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616605a9-0a96-4f8c-bc53-ec9dfba90f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:57.039607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-616605a9-0a96-4f8c-bc53-ec9dfba90f14 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:57.041658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-616605a9-0a96-4f8c-bc53-ec9dfba90f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:57.058075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:57.141675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 441/883] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:57.164260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847fa26-3cc8-4de0-b129-d1ba0cba5363 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:57.255813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:57.273441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847fa26-3cc8-4de0-b129-d1ba0cba5363 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:57.307693 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91375ed9-d3b2-4cdc-a426-98deb3f0f18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:57.308488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91375ed9-d3b2-4cdc-a426-98deb3f0f18d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:57.308772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91375ed9-d3b2-4cdc-a426-98deb3f0f18d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:57.309047 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91375ed9-d3b2-4cdc-a426-98deb3f0f18d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:57.309697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91375ed9-d3b2-4cdc-a426-98deb3f0f18d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:57.311079 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91375ed9-d3b2-4cdc-a426-98deb3f0f18d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-91375ed9-d3b2-4cdc-a426-98deb3f0f18d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:57.323440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:57.324364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:57.325154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 443/884] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:57.332267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616605a9-0a96-4f8c-bc53-ec9dfba90f14 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:57.449096 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616605a9-0a96-4f8c-bc53-ec9dfba90f14 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:57.491070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352758c4-cb4b-46c5-a7d6-f74c263a3267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:57.491743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352758c4-cb4b-46c5-a7d6-f74c263a3267 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:57.492027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352758c4-cb4b-46c5-a7d6-f74c263a3267 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:57.492303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352758c4-cb4b-46c5-a7d6-f74c263a3267 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:57.493014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352758c4-cb4b-46c5-a7d6-f74c263a3267 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:57.493323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352758c4-cb4b-46c5-a7d6-f74c263a3267 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:57.495305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352758c4-cb4b-46c5-a7d6-f74c263a3267 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:57.501532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 442/885] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:57 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:57.507313 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91375ed9-d3b2-4cdc-a426-98deb3f0f18d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:57.615936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91375ed9-d3b2-4cdc-a426-98deb3f0f18d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:57.650252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0ea02-85bb-4856-9085-6d80ea2d156f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:57.650947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0ea02-85bb-4856-9085-6d80ea2d156f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:57.651230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0ea02-85bb-4856-9085-6d80ea2d156f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:57.651508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab0ea02-85bb-4856-9085-6d80ea2d156f None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:57.652209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0ea02-85bb-4856-9085-6d80ea2d156f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:57.652520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0ea02-85bb-4856-9085-6d80ea2d156f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:57.653880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab0ea02-85bb-4856-9085-6d80ea2d156f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:57.659244 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 444/886] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:52:57 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:57.664814 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352758c4-cb4b-46c5-a7d6-f74c263a3267 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:57.677941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:57.769473 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352758c4-cb4b-46c5-a7d6-f74c263a3267 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:57.804295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92396c8-5e65-4a31-b7b1-60a4b64519c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:57.805285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92396c8-5e65-4a31-b7b1-60a4b64519c9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:57.805776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92396c8-5e65-4a31-b7b1-60a4b64519c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:57.806278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92396c8-5e65-4a31-b7b1-60a4b64519c9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:57.807819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92396c8-5e65-4a31-b7b1-60a4b64519c9 None admin] RBAC: Adding query filter params (interface=public, service_id=9b8058185a994c8c8cf384622687c576, region_id=RegionOne) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:57.808328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92396c8-5e65-4a31-b7b1-60a4b64519c9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:57.809892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92396c8-5e65-4a31-b7b1-60a4b64519c9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:57.815564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 443/887] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Jun 26 02:52:57 2022] GET /identity/v3/endpoints?service_id=9b8058185a994c8c8cf384622687c576&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:57.821218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0ea02-85bb-4856-9085-6d80ea2d156f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:57.840302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:57.931136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab0ea02-85bb-4856-9085-6d80ea2d156f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:57.965739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175db45-aaa8-41bd-b546-27ae84ad0f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:57.966434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175db45-aaa8-41bd-b546-27ae84ad0f59 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:57.966716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175db45-aaa8-41bd-b546-27ae84ad0f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:57.966990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175db45-aaa8-41bd-b546-27ae84ad0f59 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:57.967674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175db45-aaa8-41bd-b546-27ae84ad0f59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:57.969036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175db45-aaa8-41bd-b546-27ae84ad0f59 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:57.975402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 445/888] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Jun 26 02:52:57 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 02:52:58.395627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee480c4d-059f-4a44-9390-b1bb7cc78d71 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:58.396145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee480c4d-059f-4a44-9390-b1bb7cc78d71 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:58.396145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee480c4d-059f-4a44-9390-b1bb7cc78d71 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:58.397787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 444/889] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:58.403151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa6212-b946-48f5-b0cd-d625bbd7a1a4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:58.403538 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa6212-b946-48f5-b0cd-d625bbd7a1a4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:58.403717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaa6212-b946-48f5-b0cd-d625bbd7a1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:58.424932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dbaa6212-b946-48f5-b0cd-d625bbd7a1a4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:58.426157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dbaa6212-b946-48f5-b0cd-d625bbd7a1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:58.462143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9efd3-338b-4828-9666-2bc12c7d8cf6 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:58.462466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9efd3-338b-4828-9666-2bc12c7d8cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:58.462714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de9efd3-338b-4828-9666-2bc12c7d8cf6 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:58.464412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 445/890] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:52:58.469320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5436bff-3158-461f-8070-cc9a3ddc6ea8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:58.469611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5436bff-3158-461f-8070-cc9a3ddc6ea8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:58.469892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5436bff-3158-461f-8070-cc9a3ddc6ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:58.488461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b5436bff-3158-461f-8070-cc9a3ddc6ea8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:58.489644 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b5436bff-3158-461f-8070-cc9a3ddc6ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:58.501070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 446/891] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:58.524770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa6212-b946-48f5-b0cd-d625bbd7a1a4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:58.533272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:58.570005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:58.576697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 446/892] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:52:58.598795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5436bff-3158-461f-8070-cc9a3ddc6ea8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:58.638513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaa6212-b946-48f5-b0cd-d625bbd7a1a4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:58.675612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4821dbde-37a0-4129-b606-2f1d8562ffeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:58.676549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821dbde-37a0-4129-b606-2f1d8562ffeb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:58.676956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821dbde-37a0-4129-b606-2f1d8562ffeb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:58.677333 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4821dbde-37a0-4129-b606-2f1d8562ffeb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:58.688546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4821dbde-37a0-4129-b606-2f1d8562ffeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-4821dbde-37a0-4129-b606-2f1d8562ffeb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:52:58.697170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:58.698338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:52:58.699375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:58.700144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5436bff-3158-461f-8070-cc9a3ddc6ea8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:58.701621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 447/893] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Jun 26 02:52:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:52:58.703289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359d6d4-ec88-49f6-927a-88e8b99bf9dc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:58.703580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359d6d4-ec88-49f6-927a-88e8b99bf9dc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:58.703856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0359d6d4-ec88-49f6-927a-88e8b99bf9dc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:58.706468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 448/894] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:52:58.707204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359d6d4-ec88-49f6-927a-88e8b99bf9dc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:58.736383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4dd926-ee6b-4d3f-889d-944d7293564e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:58.737049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd926-ee6b-4d3f-889d-944d7293564e None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:58.737366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd926-ee6b-4d3f-889d-944d7293564e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:58.737599 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4dd926-ee6b-4d3f-889d-944d7293564e None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:58.738346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dd926-ee6b-4d3f-889d-944d7293564e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:58.739701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4dd926-ee6b-4d3f-889d-944d7293564e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:58.753411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 447/895] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:52:58 2022] POST /identity/v3/services => generated 241 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:52:58.754849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff7d8a-dfa8-4eb7-a2b4-4e00163c948e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:58.755145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff7d8a-dfa8-4eb7-a2b4-4e00163c948e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:58.755441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff7d8a-dfa8-4eb7-a2b4-4e00163c948e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:58.779958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-42ff7d8a-dfa8-4eb7-a2b4-4e00163c948e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:52:58.781141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-42ff7d8a-dfa8-4eb7-a2b4-4e00163c948e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:52:58.791795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:58.844722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0359d6d4-ec88-49f6-927a-88e8b99bf9dc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:58.846982 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:58.879318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 448/896] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:52:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:52:58.892159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f25268-f200-4c44-83cd-905d83fbad99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:58.893097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f25268-f200-4c44-83cd-905d83fbad99 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:58.893526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f25268-f200-4c44-83cd-905d83fbad99 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:58.893988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f25268-f200-4c44-83cd-905d83fbad99 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:58.894970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f25268-f200-4c44-83cd-905d83fbad99 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:58.895453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f25268-f200-4c44-83cd-905d83fbad99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:58.897045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f25268-f200-4c44-83cd-905d83fbad99 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:58.902562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff7d8a-dfa8-4eb7-a2b4-4e00163c948e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:58.913112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 449/897] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Jun 26 02:52:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:58.920034 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f25268-f200-4c44-83cd-905d83fbad99 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:59.022329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ff7d8a-dfa8-4eb7-a2b4-4e00163c948e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:59.051791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f25268-f200-4c44-83cd-905d83fbad99 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:59.059903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326aa6fa-5c56-41b9-ab02-0ab4dbbafe6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:59.060715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aa6fa-5c56-41b9-ab02-0ab4dbbafe6f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:59.061001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aa6fa-5c56-41b9-ab02-0ab4dbbafe6f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:59.061278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326aa6fa-5c56-41b9-ab02-0ab4dbbafe6f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:59.061937 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326aa6fa-5c56-41b9-ab02-0ab4dbbafe6f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:59.063344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326aa6fa-5c56-41b9-ab02-0ab4dbbafe6f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-326aa6fa-5c56-41b9-ab02-0ab4dbbafe6f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:59.076292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:59.077254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 26 02:52:59.078108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:52:59.078898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:59.082492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 449/898] 10.222.0.13 () {56 vars in 1132 bytes} [Sun Jun 26 02:52:58 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:59.084193 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326aa6fa-5c56-41b9-ab02-0ab4dbbafe6f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:59.092144 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a0ca1-8596-437c-9622-57c609a35dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:59.092939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a0ca1-8596-437c-9622-57c609a35dcd None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:59.093221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a0ca1-8596-437c-9622-57c609a35dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:59.093497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a0ca1-8596-437c-9622-57c609a35dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:59.111368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a0ca1-8596-437c-9622-57c609a35dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:59.112852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092a0ca1-8596-437c-9622-57c609a35dcd None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-092a0ca1-8596-437c-9622-57c609a35dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:52:59.139968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:59.140906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:59.141767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:52:59.141767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:52:59.141767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:52:59.141767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:59.144468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 450/899] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:52:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:59.150088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a0ca1-8596-437c-9622-57c609a35dcd None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:59.152935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:59.207259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326aa6fa-5c56-41b9-ab02-0ab4dbbafe6f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:59.251981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512dc9-3c5e-4fe4-9638-4db6b95b46da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:59.253130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512dc9-3c5e-4fe4-9638-4db6b95b46da None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:59.253800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512dc9-3c5e-4fe4-9638-4db6b95b46da None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:59.254280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0512dc9-3c5e-4fe4-9638-4db6b95b46da None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:59.255546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0512dc9-3c5e-4fe4-9638-4db6b95b46da None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:59.256268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0512dc9-3c5e-4fe4-9638-4db6b95b46da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:59.258418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0512dc9-3c5e-4fe4-9638-4db6b95b46da None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:59.261640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a0ca1-8596-437c-9622-57c609a35dcd None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:59.266082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 450/900] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:52:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:59.274352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512dc9-3c5e-4fe4-9638-4db6b95b46da None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:59.307573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d3171-5df8-44f3-bc39-30f432b6cd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:59.308346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d3171-5df8-44f3-bc39-30f432b6cd1e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:59.308696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d3171-5df8-44f3-bc39-30f432b6cd1e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:59.309044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7d3171-5df8-44f3-bc39-30f432b6cd1e None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:59.309840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d3171-5df8-44f3-bc39-30f432b6cd1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:59.310478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d3171-5df8-44f3-bc39-30f432b6cd1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:59.312817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7d3171-5df8-44f3-bc39-30f432b6cd1e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:59.329024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 451/901] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:52:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:59.334530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d3171-5df8-44f3-bc39-30f432b6cd1e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:59.379230 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0512dc9-3c5e-4fe4-9638-4db6b95b46da None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:59.381071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:59.397980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:59.417148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2237401-598f-4525-832a-60ef43ce0ecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:59.417839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2237401-598f-4525-832a-60ef43ce0ecf None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:59.418385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2237401-598f-4525-832a-60ef43ce0ecf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:59.418665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2237401-598f-4525-832a-60ef43ce0ecf None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:59.419367 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2237401-598f-4525-832a-60ef43ce0ecf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:59.419672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2237401-598f-4525-832a-60ef43ce0ecf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:59.421468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2237401-598f-4525-832a-60ef43ce0ecf None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:59.432268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 451/902] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:52:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:59.434268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2237401-598f-4525-832a-60ef43ce0ecf None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:59.464271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7d3171-5df8-44f3-bc39-30f432b6cd1e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:59.499406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8277d74-5e40-4dac-9e87-51417a5be23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:59.500472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277d74-5e40-4dac-9e87-51417a5be23f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:59.500584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277d74-5e40-4dac-9e87-51417a5be23f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:59.500775 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8277d74-5e40-4dac-9e87-51417a5be23f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:59.507978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8277d74-5e40-4dac-9e87-51417a5be23f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:59.509412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8277d74-5e40-4dac-9e87-51417a5be23f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b8277d74-5e40-4dac-9e87-51417a5be23f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:52:59.525039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:52:59.526469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:52:59.527942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 452/903] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Jun 26 02:52:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:52:59.531890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8277d74-5e40-4dac-9e87-51417a5be23f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:59.550738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2237401-598f-4525-832a-60ef43ce0ecf None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:59.589145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d1529-3fed-4818-a249-d83b262d3bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:59.590309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d1529-3fed-4818-a249-d83b262d3bf7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:59.590848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d1529-3fed-4818-a249-d83b262d3bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:59.591345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22d1529-3fed-4818-a249-d83b262d3bf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:59.592338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d1529-3fed-4818-a249-d83b262d3bf7 None admin] RBAC: Adding query filter params (interface=public, service_id=3ab924ce0e474fc1a8cb307c9df6f138, region_id=RegionOne) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:59.593025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d1529-3fed-4818-a249-d83b262d3bf7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:59.594749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22d1529-3fed-4818-a249-d83b262d3bf7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:59.605329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 452/904] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Jun 26 02:52:59 2022] GET /identity/v3/endpoints?service_id=3ab924ce0e474fc1a8cb307c9df6f138&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:52:59.606879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d1529-3fed-4818-a249-d83b262d3bf7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:59.630722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:59.637756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8277d74-5e40-4dac-9e87-51417a5be23f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:59.660568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:59.673071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ca92b-2222-4f45-b884-bfb9bedc86d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:59.673959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ca92b-2222-4f45-b884-bfb9bedc86d8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:59.674282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ca92b-2222-4f45-b884-bfb9bedc86d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:59.674594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2ca92b-2222-4f45-b884-bfb9bedc86d8 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:59.675576 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ca92b-2222-4f45-b884-bfb9bedc86d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:52:59.675938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ca92b-2222-4f45-b884-bfb9bedc86d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:59.677290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2ca92b-2222-4f45-b884-bfb9bedc86d8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:59.695562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 453/905] 10.222.0.13 () {56 vars in 1149 bytes} [Sun Jun 26 02:52:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:52:59.701290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ca92b-2222-4f45-b884-bfb9bedc86d8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:52:59.722429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22d1529-3fed-4818-a249-d83b262d3bf7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:59.763335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887d497-09ff-4bf7-8f47-d5b7045d04a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:59.764441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887d497-09ff-4bf7-8f47-d5b7045d04a9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:59.764977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887d497-09ff-4bf7-8f47-d5b7045d04a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:59.765457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887d497-09ff-4bf7-8f47-d5b7045d04a9 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:59.766403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887d497-09ff-4bf7-8f47-d5b7045d04a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:59.768209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887d497-09ff-4bf7-8f47-d5b7045d04a9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:52:59.782315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 453/906] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Jun 26 02:52:59 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 02:52:59.813228 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2ca92b-2222-4f45-b884-bfb9bedc86d8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:52:59.861026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f64db-e422-49db-bf47-c6da70977e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:52:59.862029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f64db-e422-49db-bf47-c6da70977e5f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:52:59.862543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f64db-e422-49db-bf47-c6da70977e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:52:59.863038 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f64db-e422-49db-bf47-c6da70977e5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:52:59.902456 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:52:59.959809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f64db-e422-49db-bf47-c6da70977e5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:52:59.961269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f64db-e422-49db-bf47-c6da70977e5f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:00.065002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 454/907] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:52:59 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:53:00.070803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887d497-09ff-4bf7-8f47-d5b7045d04a9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:00.080069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2dfbcc-091e-47dd-9102-8b3847f760d0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.080629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2dfbcc-091e-47dd-9102-8b3847f760d0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.080909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2dfbcc-091e-47dd-9102-8b3847f760d0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.082597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 455/908] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:00.087408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1018843-3d0a-4c04-9e6d-1b62fa3417d6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.087691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1018843-3d0a-4c04-9e6d-1b62fa3417d6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.087999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1018843-3d0a-4c04-9e6d-1b62fa3417d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.104662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f1018843-3d0a-4c04-9e6d-1b62fa3417d6 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:00.105835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f1018843-3d0a-4c04-9e6d-1b62fa3417d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:00.160876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:00.185232 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 456/909] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:00.196018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887d497-09ff-4bf7-8f47-d5b7045d04a9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:00.205465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:00.207076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1018843-3d0a-4c04-9e6d-1b62fa3417d6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:00.239871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eac35b-7edc-4bdf-98ee-42681f9c3fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:00.241502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eac35b-7edc-4bdf-98ee-42681f9c3fda None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.242261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eac35b-7edc-4bdf-98ee-42681f9c3fda None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.242748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eac35b-7edc-4bdf-98ee-42681f9c3fda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.257238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eac35b-7edc-4bdf-98ee-42681f9c3fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:00.259677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eac35b-7edc-4bdf-98ee-42681f9c3fda None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b3eac35b-7edc-4bdf-98ee-42681f9c3fda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:00.284296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:00.285221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:00.286126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:00.286126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:00.286126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:00.286126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:00.288910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 454/910] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:00.297071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eac35b-7edc-4bdf-98ee-42681f9c3fda None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:00.319989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1018843-3d0a-4c04-9e6d-1b62fa3417d6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:00.358736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6baa82-f62c-4762-aeec-fc91d0fd07ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:00.359815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6baa82-f62c-4762-aeec-fc91d0fd07ef None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.359815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6baa82-f62c-4762-aeec-fc91d0fd07ef None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.360125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6baa82-f62c-4762-aeec-fc91d0fd07ef None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.360829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6baa82-f62c-4762-aeec-fc91d0fd07ef None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:00.362228 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6baa82-f62c-4762-aeec-fc91d0fd07ef None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-dd6baa82-f62c-4762-aeec-fc91d0fd07ef None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:00.374530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:00.375527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 26 02:53:00.376341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:00.380618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 457/911] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:53:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:00.381573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6baa82-f62c-4762-aeec-fc91d0fd07ef None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:00.435407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:00.435905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eac35b-7edc-4bdf-98ee-42681f9c3fda None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:00.474449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906692c-9e46-4f93-98a4-e5eed24cbfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:00.475420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906692c-9e46-4f93-98a4-e5eed24cbfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.475887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906692c-9e46-4f93-98a4-e5eed24cbfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.476340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c906692c-9e46-4f93-98a4-e5eed24cbfa7 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.477270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906692c-9e46-4f93-98a4-e5eed24cbfa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:00.477757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906692c-9e46-4f93-98a4-e5eed24cbfa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:00.479912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c906692c-9e46-4f93-98a4-e5eed24cbfa7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:00.490654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6baa82-f62c-4762-aeec-fc91d0fd07ef None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:00.498022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 455/912] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:00.499523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906692c-9e46-4f93-98a4-e5eed24cbfa7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:00.524983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:00.526695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ea7fc-ddfe-4870-981e-ebf83ef7080d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:00.527299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ea7fc-ddfe-4870-981e-ebf83ef7080d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.527576 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ea7fc-ddfe-4870-981e-ebf83ef7080d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.527888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49ea7fc-ddfe-4870-981e-ebf83ef7080d None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.528617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ea7fc-ddfe-4870-981e-ebf83ef7080d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:00.528868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ea7fc-ddfe-4870-981e-ebf83ef7080d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:00.530265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49ea7fc-ddfe-4870-981e-ebf83ef7080d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:00.540818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 458/913] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:53:00 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:00.542279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ea7fc-ddfe-4870-981e-ebf83ef7080d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:00.610226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c906692c-9e46-4f93-98a4-e5eed24cbfa7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:00.646494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8f17-5090-4922-bae2-d464821bd80d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:00.647626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8f17-5090-4922-bae2-d464821bd80d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.648117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8f17-5090-4922-bae2-d464821bd80d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.648639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c8f17-5090-4922-bae2-d464821bd80d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.651716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49ea7fc-ddfe-4870-981e-ebf83ef7080d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:00.656770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8f17-5090-4922-bae2-d464821bd80d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:00.658468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880c8f17-5090-4922-bae2-d464821bd80d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:00.664856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-880c8f17-5090-4922-bae2-d464821bd80d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:53:00.673040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:00.673933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:00.675025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:53:00.675025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:53:00.675025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:53:00.675025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:53:00.675025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:53:00.675025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:53:00.675025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:00.675025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:00.681514 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 456/914] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:53:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:00.682586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8f17-5090-4922-bae2-d464821bd80d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:00.687857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f670cf-f27f-44fd-a191-7cc38e07cd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:00.688418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f670cf-f27f-44fd-a191-7cc38e07cd01 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.688704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f670cf-f27f-44fd-a191-7cc38e07cd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.688981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f670cf-f27f-44fd-a191-7cc38e07cd01 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.689685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f670cf-f27f-44fd-a191-7cc38e07cd01 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:00.689992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f670cf-f27f-44fd-a191-7cc38e07cd01 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:00.691389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f670cf-f27f-44fd-a191-7cc38e07cd01 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:00.701716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 459/915] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Jun 26 02:53:00 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:00.702541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f670cf-f27f-44fd-a191-7cc38e07cd01 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:00.769863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:00.791239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880c8f17-5090-4922-bae2-d464821bd80d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:00.808691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f670cf-f27f-44fd-a191-7cc38e07cd01 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:00.829764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9298b-f8fb-408e-a9fb-9970f97dc973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:00.830696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9298b-f8fb-408e-a9fb-9970f97dc973 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.831158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9298b-f8fb-408e-a9fb-9970f97dc973 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.831582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e9298b-f8fb-408e-a9fb-9970f97dc973 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.832517 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9298b-f8fb-408e-a9fb-9970f97dc973 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:00.832985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9298b-f8fb-408e-a9fb-9970f97dc973 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:00.834596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e9298b-f8fb-408e-a9fb-9970f97dc973 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:00.845312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b5f5c-54b4-425e-847b-4b3a0fc577d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:00.845763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b5f5c-54b4-425e-847b-4b3a0fc577d6 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.846064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b5f5c-54b4-425e-847b-4b3a0fc577d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.846344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b5f5c-54b4-425e-847b-4b3a0fc577d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.847078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b5f5c-54b4-425e-847b-4b3a0fc577d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:00.854328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b5f5c-54b4-425e-847b-4b3a0fc577d6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:00.854736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 457/916] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:53:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:00.855766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9298b-f8fb-408e-a9fb-9970f97dc973 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:00.877989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 460/917] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Jun 26 02:53:00 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:53:00.882103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd6b27-3f2c-4669-a719-a90891c88e96 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.882103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd6b27-3f2c-4669-a719-a90891c88e96 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.882103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bd6b27-3f2c-4669-a719-a90891c88e96 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.882103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 461/918] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:00.887087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5efdba-b19a-43f0-8377-0de3e8764399 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:00.887381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5efdba-b19a-43f0-8377-0de3e8764399 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:00.887662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5efdba-b19a-43f0-8377-0de3e8764399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:00.906119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8a5efdba-b19a-43f0-8377-0de3e8764399 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:00.907355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8a5efdba-b19a-43f0-8377-0de3e8764399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:00.943097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:00.993806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e9298b-f8fb-408e-a9fb-9970f97dc973 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:01.003797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 462/919] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:01.032725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5efdba-b19a-43f0-8377-0de3e8764399 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:01.044967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d7e37-45a4-404e-be47-89d50b21eae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:01.046486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d7e37-45a4-404e-be47-89d50b21eae9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:01.047261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d7e37-45a4-404e-be47-89d50b21eae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:01.047761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42d7e37-45a4-404e-be47-89d50b21eae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:01.049088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d7e37-45a4-404e-be47-89d50b21eae9 None admin] RBAC: Adding query filter params (role.id=d34bc405b0c04033a57dc7dae296fdee, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=828fd11b79d045dc85e0973cd255bcf7) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:01.050088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d7e37-45a4-404e-be47-89d50b21eae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:01.051821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42d7e37-45a4-404e-be47-89d50b21eae9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:01.059708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 458/920] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:53:00 2022] GET /identity/v3/role_assignments?user.id=828fd11b79d045dc85e0973cd255bcf7&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=d34bc405b0c04033a57dc7dae296fdee => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:01.157181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5efdba-b19a-43f0-8377-0de3e8764399 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:01.199893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3faf-0e6f-485d-836c-1dea64563cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:01.200735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3faf-0e6f-485d-836c-1dea64563cbf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:01.201022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3faf-0e6f-485d-836c-1dea64563cbf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:01.201303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c3faf-0e6f-485d-836c-1dea64563cbf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:01.201973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c3faf-0e6f-485d-836c-1dea64563cbf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:01.203396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c3faf-0e6f-485d-836c-1dea64563cbf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-c94c3faf-0e6f-485d-836c-1dea64563cbf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:01.216545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:01.217595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:01.218436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 463/921] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:53:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:01.223949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d7e37-45a4-404e-be47-89d50b21eae9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:01.238034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:01.339404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42d7e37-45a4-404e-be47-89d50b21eae9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:01.378078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b4466-0324-4fbd-904b-e66b8bd0ede5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:01.379129 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b4466-0324-4fbd-904b-e66b8bd0ede5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:01.379613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b4466-0324-4fbd-904b-e66b8bd0ede5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:01.380085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b4466-0324-4fbd-904b-e66b8bd0ede5 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:01.381000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b4466-0324-4fbd-904b-e66b8bd0ede5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:01.381535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b4466-0324-4fbd-904b-e66b8bd0ede5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:01.383113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92b4466-0324-4fbd-904b-e66b8bd0ede5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:01.388640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 459/922] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Jun 26 02:53:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:01.394414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3faf-0e6f-485d-836c-1dea64563cbf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:01.420204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:01.508413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c3faf-0e6f-485d-836c-1dea64563cbf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:01.543816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6518b8a-0785-43ef-9b55-8132a2f462b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:01.544496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6518b8a-0785-43ef-9b55-8132a2f462b5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:01.544834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6518b8a-0785-43ef-9b55-8132a2f462b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:01.545119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6518b8a-0785-43ef-9b55-8132a2f462b5 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:01.545841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6518b8a-0785-43ef-9b55-8132a2f462b5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:01.546182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6518b8a-0785-43ef-9b55-8132a2f462b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:01.547557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6518b8a-0785-43ef-9b55-8132a2f462b5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:01.553200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 464/923] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Jun 26 02:53:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:01.559229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b4466-0324-4fbd-904b-e66b8bd0ede5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:01.632994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:01.669008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92b4466-0324-4fbd-904b-e66b8bd0ede5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:01.704850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f56e6-39a9-46f9-8234-3528d30fba07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:01.705853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f56e6-39a9-46f9-8234-3528d30fba07 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:01.706368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f56e6-39a9-46f9-8234-3528d30fba07 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:01.706849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f56e6-39a9-46f9-8234-3528d30fba07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:01.707862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f56e6-39a9-46f9-8234-3528d30fba07 None admin] RBAC: Adding query filter params (interface=public, service_id=1592033d11d043eaa54ea9e8460f9918, region_id=RegionOne) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:01.708377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f56e6-39a9-46f9-8234-3528d30fba07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:01.710029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f56e6-39a9-46f9-8234-3528d30fba07 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:01.715863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 460/924] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Jun 26 02:53:01 2022] GET /identity/v3/endpoints?service_id=1592033d11d043eaa54ea9e8460f9918&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:01.721502 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6518b8a-0785-43ef-9b55-8132a2f462b5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:01.814417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:01.831100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6518b8a-0785-43ef-9b55-8132a2f462b5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:01.873709 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611f3d8-9a8b-48ab-b10f-cbc66c9293af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:01.874677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f3d8-9a8b-48ab-b10f-cbc66c9293af None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:01.874958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f3d8-9a8b-48ab-b10f-cbc66c9293af None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:01.875233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611f3d8-9a8b-48ab-b10f-cbc66c9293af None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:01.875914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611f3d8-9a8b-48ab-b10f-cbc66c9293af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:01.877262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611f3d8-9a8b-48ab-b10f-cbc66c9293af None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:01.883931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 465/925] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Jun 26 02:53:01 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 02:53:01.909281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939448d5-dd14-456f-8c68-78bdb861bb15 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:01.909640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939448d5-dd14-456f-8c68-78bdb861bb15 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:01.909846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939448d5-dd14-456f-8c68-78bdb861bb15 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:01.912752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 461/926] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:01.917489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd80681-15c1-45cd-afb0-b8787ec57a1f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:01.917778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd80681-15c1-45cd-afb0-b8787ec57a1f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:01.918068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd80681-15c1-45cd-afb0-b8787ec57a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:01.936100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cfd80681-15c1-45cd-afb0-b8787ec57a1f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:01.937334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cfd80681-15c1-45cd-afb0-b8787ec57a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:02.018855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 466/927] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:02.041884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939448d5-dd14-456f-8c68-78bdb861bb15 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:02.161804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939448d5-dd14-456f-8c68-78bdb861bb15 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:02.196550 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b3182-b904-4b49-b2ce-c34403f96e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:02.197391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b3182-b904-4b49-b2ce-c34403f96e7d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:02.197675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b3182-b904-4b49-b2ce-c34403f96e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:02.197952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0b3182-b904-4b49-b2ce-c34403f96e7d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:02.198631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b3182-b904-4b49-b2ce-c34403f96e7d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:02.199987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0b3182-b904-4b49-b2ce-c34403f96e7d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:02.201777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ed0b3182-b904-4b49-b2ce-c34403f96e7d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:02.214459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:02.215642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 26 02:53:02.216671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:02.220546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 462/928] 10.222.0.13 () {56 vars in 1132 bytes} [Sun Jun 26 02:53:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:02.221917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd80681-15c1-45cd-afb0-b8787ec57a1f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:02.309801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:02.329545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd80681-15c1-45cd-afb0-b8787ec57a1f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:02.364130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5668f16-4c1a-4507-97f3-b310c6cf7803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:02.364796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5668f16-4c1a-4507-97f3-b310c6cf7803 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:02.365078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5668f16-4c1a-4507-97f3-b310c6cf7803 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:02.365350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5668f16-4c1a-4507-97f3-b310c6cf7803 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:02.366079 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5668f16-4c1a-4507-97f3-b310c6cf7803 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:02.366382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5668f16-4c1a-4507-97f3-b310c6cf7803 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:02.367738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5668f16-4c1a-4507-97f3-b310c6cf7803 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:02.373570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 467/929] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:53:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:02.379315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b3182-b904-4b49-b2ce-c34403f96e7d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:02.482605 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0b3182-b904-4b49-b2ce-c34403f96e7d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:02.517693 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d50bc6-eb0d-4e9b-b3e1-c98ec0d6821c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:02.518389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d50bc6-eb0d-4e9b-b3e1-c98ec0d6821c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:02.518675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d50bc6-eb0d-4e9b-b3e1-c98ec0d6821c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:02.518953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d50bc6-eb0d-4e9b-b3e1-c98ec0d6821c None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:02.519657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d50bc6-eb0d-4e9b-b3e1-c98ec0d6821c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:02.519966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d50bc6-eb0d-4e9b-b3e1-c98ec0d6821c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:02.521322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d50bc6-eb0d-4e9b-b3e1-c98ec0d6821c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:02.526682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 463/930] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:53:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:02.532319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5668f16-4c1a-4507-97f3-b310c6cf7803 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:02.635608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5668f16-4c1a-4507-97f3-b310c6cf7803 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:02.674737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cb555d-b52f-4806-a882-279d5e212d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:02.675397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb555d-b52f-4806-a882-279d5e212d9e None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:02.675677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb555d-b52f-4806-a882-279d5e212d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:02.675951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cb555d-b52f-4806-a882-279d5e212d9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:02.676683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb555d-b52f-4806-a882-279d5e212d9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:02.678031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cb555d-b52f-4806-a882-279d5e212d9e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:02.694951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:02.700748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 468/931] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Jun 26 02:53:02 2022] POST /identity/v3/endpoints => generated 347 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:53:03.107983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d91fd-8739-42ba-9486-8bb1eda01c6b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:03.108467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d91fd-8739-42ba-9486-8bb1eda01c6b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:03.108467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d91fd-8739-42ba-9486-8bb1eda01c6b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:03.110121 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 464/932] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:03.115274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e13b98-b5c3-42e4-805c-92bd00d349ff None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:03.115567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e13b98-b5c3-42e4-805c-92bd00d349ff None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:03.115846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e13b98-b5c3-42e4-805c-92bd00d349ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:03.132971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2e13b98-b5c3-42e4-805c-92bd00d349ff None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:03.134196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2e13b98-b5c3-42e4-805c-92bd00d349ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:03.210024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 469/933] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:03.233509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d91fd-8739-42ba-9486-8bb1eda01c6b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:03.295917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0be9ea-62b2-4ec9-b9ac-3e570fa41f62 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:03.296218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0be9ea-62b2-4ec9-b9ac-3e570fa41f62 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:03.296477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0be9ea-62b2-4ec9-b9ac-3e570fa41f62 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:03.298981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:03.301585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 470/934] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:03.302833 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4050e-c9ab-439b-9171-a19dd4f44a4a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:03.303118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4050e-c9ab-439b-9171-a19dd4f44a4a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:03.303394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4050e-c9ab-439b-9171-a19dd4f44a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:03.320367 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-40b4050e-c9ab-439b-9171-a19dd4f44a4a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:03.321543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-40b4050e-c9ab-439b-9171-a19dd4f44a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:03.344446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d91fd-8739-42ba-9486-8bb1eda01c6b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:03.381824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:03.382867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:03.383480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:03.384110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:03.385048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:03.386986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:03.388619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WARNING keystone.server.flask.common [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 26 02:53:03.398371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 471/935] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:03.408556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.sql.core [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 26 02:53:03.408556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 26 02:53:03.408556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [parameters: {'id': '7878d1aa66b3413eb93e5ae219fde4fc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 26 02:53:03.408556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80103) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 26 02:53:03.415226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4050e-c9ab-439b-9171-a19dd4f44a4a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7878d1aa66b3413eb93e5ae219fde4fc.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7878d1aa66b3413eb93e5ae219fde4fc. Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = self._query(query) Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application conn.query(q) Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 26 02:53:03.418596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result.read() Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 26 02:53:03.419630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application next(self.gen) Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application yield resource Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application next(self.gen) Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.session.rollback() Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.force_reraise() Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise self.value Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application session.commit() Jun 26 02:53:03.420601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._prepare_impl() Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.session.flush() Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._flush(objects) Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application compat.raise_( Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application flush_context.execute() Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 26 02:53:03.421531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rec.execute(self) Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 26 02:53:03.422509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = self._query(query) Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application conn.query(q) Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result.read() Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 26 02:53:03.424396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [parameters: {'id': '7878d1aa66b3413eb93e5ae219fde4fc', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:03.426687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:03.428668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:03.430886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:03.430886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 26 02:53:03.430886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 26 02:53:03.430886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:03.430886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:03.430886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 26 02:53:03.430886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 26 02:53:03.430886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 7878d1aa66b3413eb93e5ae219fde4fc. Jun 26 02:53:03.430886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:03.435000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 465/936] 10.222.0.13 () {60 vars in 1140 bytes} [Sun Jun 26 02:53:03 2022] POST /identity/v3/projects => generated 262 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:03.441276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:03.474621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:03.521372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4050e-c9ab-439b-9171-a19dd4f44a4a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:03.557945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5687a071-4c52-4845-b4fb-6616eb2ebb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:03.558793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687a071-4c52-4845-b4fb-6616eb2ebb32 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:03.559072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687a071-4c52-4845-b4fb-6616eb2ebb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:03.559341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5687a071-4c52-4845-b4fb-6616eb2ebb32 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:03.560824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deac7a-0d63-4dea-beb3-e3a5f636402b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:03.570053 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5687a071-4c52-4845-b4fb-6616eb2ebb32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5687a071-4c52-4845-b4fb-6616eb2ebb32 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:03.579155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:03.580075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:53:03.580951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:03.585694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:03.585826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 472/937] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Jun 26 02:53:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:03.587424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5687a071-4c52-4845-b4fb-6616eb2ebb32 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:03.604513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a2eb7-97dd-43a2-a9b9-c0e15968c406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:03.605718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a2eb7-97dd-43a2-a9b9-c0e15968c406 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:03.606385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a2eb7-97dd-43a2-a9b9-c0e15968c406 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:03.606869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5a2eb7-97dd-43a2-a9b9-c0e15968c406 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:03.614621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a2eb7-97dd-43a2-a9b9-c0e15968c406 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:03.616618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5a2eb7-97dd-43a2-a9b9-c0e15968c406 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ce5a2eb7-97dd-43a2-a9b9-c0e15968c406 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:53:03.632771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:03.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:03.634552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:53:03.634552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:53:03.634552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:53:03.634552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:53:03.634552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:53:03.634552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:53:03.634552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:03.634552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:03.637593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 466/938] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:53:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:03.643425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a2eb7-97dd-43a2-a9b9-c0e15968c406 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:03.696349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5687a071-4c52-4845-b4fb-6616eb2ebb32 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:03.714337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:03.733199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f8569-457b-4574-916a-669070745788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:03.733885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f8569-457b-4574-916a-669070745788 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:03.734324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f8569-457b-4574-916a-669070745788 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:03.734575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f8569-457b-4574-916a-669070745788 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:03.735322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f8569-457b-4574-916a-669070745788 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:03.735681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f8569-457b-4574-916a-669070745788 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:03.737074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f8569-457b-4574-916a-669070745788 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:03.750134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 473/939] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Jun 26 02:53:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:03.751934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f8569-457b-4574-916a-669070745788 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:03.757789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5a2eb7-97dd-43a2-a9b9-c0e15968c406 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:03.798162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5d432-3cc7-4586-943e-81923c1361fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:03.799184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d432-3cc7-4586-943e-81923c1361fe None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:03.799784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d432-3cc7-4586-943e-81923c1361fe None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:03.800261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d432-3cc7-4586-943e-81923c1361fe None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:03.801262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5d432-3cc7-4586-943e-81923c1361fe None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:03.801788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5d432-3cc7-4586-943e-81923c1361fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:03.803853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5d432-3cc7-4586-943e-81923c1361fe None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:03.838504 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 467/940] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:53:03 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:03.869595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f8569-457b-4574-916a-669070745788 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:03.916240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ff5498-4f68-4b63-9c2d-380e8e28bac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:03.917304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff5498-4f68-4b63-9c2d-380e8e28bac8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:03.917687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff5498-4f68-4b63-9c2d-380e8e28bac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:03.918086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ff5498-4f68-4b63-9c2d-380e8e28bac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:03.930619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff5498-4f68-4b63-9c2d-380e8e28bac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:03.932809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ff5498-4f68-4b63-9c2d-380e8e28bac8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e9ff5498-4f68-4b63-9c2d-380e8e28bac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:03.952415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:03.953567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:03.954715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:03.954715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:03.954715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:03.954715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:03.956342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 474/941] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:03.961809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5d432-3cc7-4586-943e-81923c1361fe None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:04.000952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7b7c0-197d-4d75-a2a9-ca6da0514772 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.001238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7b7c0-197d-4d75-a2a9-ca6da0514772 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.001520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef7b7c0-197d-4d75-a2a9-ca6da0514772 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.004476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 475/942] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:04.005519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:04.009072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79990cf-43e3-462e-b1da-d99dd4c9e4ca None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.009365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79990cf-43e3-462e-b1da-d99dd4c9e4ca None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.009690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79990cf-43e3-462e-b1da-d99dd4c9e4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.027135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b79990cf-43e3-462e-b1da-d99dd4c9e4ca None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:04.028324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b79990cf-43e3-462e-b1da-d99dd4c9e4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:04.055489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:04.083443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5d432-3cc7-4586-943e-81923c1361fe None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:04.105948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 476/943] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:04.123271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f7a16-a368-4124-9f36-9cfd6140c094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:04.124196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f7a16-a368-4124-9f36-9cfd6140c094 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.124483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f7a16-a368-4124-9f36-9cfd6140c094 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.124758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f7a16-a368-4124-9f36-9cfd6140c094 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.125485 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f7a16-a368-4124-9f36-9cfd6140c094 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:04.125794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f7a16-a368-4124-9f36-9cfd6140c094 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:04.127197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f7a16-a368-4124-9f36-9cfd6140c094 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:04.130859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79990cf-43e3-462e-b1da-d99dd4c9e4ca None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:04.140234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 468/944] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:04.146374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f7a16-a368-4124-9f36-9cfd6140c094 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:04.262770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79990cf-43e3-462e-b1da-d99dd4c9e4ca None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:04.268498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:04.275047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f7a16-a368-4124-9f36-9cfd6140c094 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:04.310488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457508ab-11da-42cb-aa82-35517f0bd3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:04.311574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457508ab-11da-42cb-aa82-35517f0bd3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.311980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457508ab-11da-42cb-aa82-35517f0bd3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.312464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457508ab-11da-42cb-aa82-35517f0bd3e7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.313227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457508ab-11da-42cb-aa82-35517f0bd3e7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:04.314746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457508ab-11da-42cb-aa82-35517f0bd3e7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:04.318484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0889f2-4af1-466d-86cd-7caea7567721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:04.319595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0889f2-4af1-466d-86cd-7caea7567721 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.320024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0889f2-4af1-466d-86cd-7caea7567721 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.320439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0889f2-4af1-466d-86cd-7caea7567721 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-457508ab-11da-42cb-aa82-35517f0bd3e7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:04.327843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:04.328989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:04.330012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:04.331654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0889f2-4af1-466d-86cd-7caea7567721 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:04.333226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0889f2-4af1-466d-86cd-7caea7567721 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:04.337055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 477/945] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:53:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:04.338497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457508ab-11da-42cb-aa82-35517f0bd3e7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ec0889f2-4af1-466d-86cd-7caea7567721 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:53:04.351737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:04.352845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:04.354499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:04.354499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:04.354499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:53:04.354499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:53:04.354499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:53:04.354499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:53:04.354499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:53:04.354499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:53:04.354499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:53:04.354499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:04.357023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 469/946] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Jun 26 02:53:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:04.370066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0889f2-4af1-466d-86cd-7caea7567721 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:04.379616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:04.463507 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457508ab-11da-42cb-aa82-35517f0bd3e7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:04.471573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0889f2-4af1-466d-86cd-7caea7567721 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:04.503627 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec2c2b-8f77-4e46-b316-7e02aa2ce7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:04.504511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec2c2b-8f77-4e46-b316-7e02aa2ce7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.504932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec2c2b-8f77-4e46-b316-7e02aa2ce7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.505304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec2c2b-8f77-4e46-b316-7e02aa2ce7f6 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.506164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec2c2b-8f77-4e46-b316-7e02aa2ce7f6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:04.506584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec2c2b-8f77-4e46-b316-7e02aa2ce7f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:04.508055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec2c2b-8f77-4e46-b316-7e02aa2ce7f6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:04.508894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724e341-f1fa-4b87-8fef-0459551af92b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:04.509711 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724e341-f1fa-4b87-8fef-0459551af92b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.510002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724e341-f1fa-4b87-8fef-0459551af92b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.510408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7724e341-f1fa-4b87-8fef-0459551af92b None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.511261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724e341-f1fa-4b87-8fef-0459551af92b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:04.511572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724e341-f1fa-4b87-8fef-0459551af92b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:04.513046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7724e341-f1fa-4b87-8fef-0459551af92b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:04.518948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 478/947] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Jun 26 02:53:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:04.519940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec2c2b-8f77-4e46-b316-7e02aa2ce7f6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:04.536212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 470/948] 10.222.0.13 () {56 vars in 1149 bytes} [Sun Jun 26 02:53:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:04.537512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724e341-f1fa-4b87-8fef-0459551af92b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:04.557113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:04.635636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:04.649781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7724e341-f1fa-4b87-8fef-0459551af92b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:04.650788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec2c2b-8f77-4e46-b316-7e02aa2ce7f6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:04.688432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1143ed6-0602-4c2a-8632-2178fb1dbdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:04.689263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143ed6-0602-4c2a-8632-2178fb1dbdd7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.689657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143ed6-0602-4c2a-8632-2178fb1dbdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.689920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de8b05-6818-4ae7-b75b-ff737aa50126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:04.690161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1143ed6-0602-4c2a-8632-2178fb1dbdd7 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.690981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1143ed6-0602-4c2a-8632-2178fb1dbdd7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:04.691189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de8b05-6818-4ae7-b75b-ff737aa50126 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.691505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1143ed6-0602-4c2a-8632-2178fb1dbdd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:04.691756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de8b05-6818-4ae7-b75b-ff737aa50126 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.692173 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de8b05-6818-4ae7-b75b-ff737aa50126 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.693167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1143ed6-0602-4c2a-8632-2178fb1dbdd7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:04.703061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de8b05-6818-4ae7-b75b-ff737aa50126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:04.703516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 479/949] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Jun 26 02:53:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:04.704673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1143ed6-0602-4c2a-8632-2178fb1dbdd7 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:04.705272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29de8b05-6818-4ae7-b75b-ff737aa50126 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-29de8b05-6818-4ae7-b75b-ff737aa50126 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:04.724297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:04.725231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:04.726222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:04.726222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:04.726222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:04.726222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:04.728232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 471/950] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:04.734091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de8b05-6818-4ae7-b75b-ff737aa50126 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:04.806188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:04.820323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1143ed6-0602-4c2a-8632-2178fb1dbdd7 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:04.841672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de8b05-6818-4ae7-b75b-ff737aa50126 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:04.863151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c357f-3fb6-45b6-887c-a8ccfd8cdb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:04.863818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c357f-3fb6-45b6-887c-a8ccfd8cdb56 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.864109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c357f-3fb6-45b6-887c-a8ccfd8cdb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.864385 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c357f-3fb6-45b6-887c-a8ccfd8cdb56 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.865122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c357f-3fb6-45b6-887c-a8ccfd8cdb56 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:04.866540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c357f-3fb6-45b6-887c-a8ccfd8cdb56 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:04.881224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be29e6b-4369-4c8d-a262-ae2e07378cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:04.882106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29e6b-4369-4c8d-a262-ae2e07378cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.882464 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29e6b-4369-4c8d-a262-ae2e07378cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.882836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be29e6b-4369-4c8d-a262-ae2e07378cb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.883729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be29e6b-4369-4c8d-a262-ae2e07378cb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:04.884143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be29e6b-4369-4c8d-a262-ae2e07378cb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:04.885821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be29e6b-4369-4c8d-a262-ae2e07378cb3 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:04.895578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 480/951] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Jun 26 02:53:04 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:53:04.895578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aea5b5-2577-48a1-b717-ef0d14661ca8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.895578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aea5b5-2577-48a1-b717-ef0d14661ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.895578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aea5b5-2577-48a1-b717-ef0d14661ca8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.895578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 481/952] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:04.902785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df74ad0-3610-42a4-84ef-0eb67d9077f6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:04.903232 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df74ad0-3610-42a4-84ef-0eb67d9077f6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:04.903611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 472/953] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:04.903730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df74ad0-3610-42a4-84ef-0eb67d9077f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:04.909252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be29e6b-4369-4c8d-a262-ae2e07378cb3 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:04.924404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6df74ad0-3610-42a4-84ef-0eb67d9077f6 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:04.926409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6df74ad0-3610-42a4-84ef-0eb67d9077f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:04.929685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:05.031603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 482/954] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:05.047018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be29e6b-4369-4c8d-a262-ae2e07378cb3 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:05.055035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df74ad0-3610-42a4-84ef-0eb67d9077f6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:05.077666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:05.089691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89095460-e35b-46d1-a893-38666a81adf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:05.090613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89095460-e35b-46d1-a893-38666a81adf7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:05.090976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89095460-e35b-46d1-a893-38666a81adf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:05.091308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89095460-e35b-46d1-a893-38666a81adf7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:05.099033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89095460-e35b-46d1-a893-38666a81adf7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:05.100563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89095460-e35b-46d1-a893-38666a81adf7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-89095460-e35b-46d1-a893-38666a81adf7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:53:05.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:05.117257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:05.118370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:53:05.118370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:53:05.118370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:53:05.118370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:53:05.118370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:53:05.118370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:53:05.118370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:05.118370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:05.119756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 473/955] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:53:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:05.126229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89095460-e35b-46d1-a893-38666a81adf7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:05.176519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df74ad0-3610-42a4-84ef-0eb67d9077f6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:05.224007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117ce8c-3f45-4aff-a630-620a2929b183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:05.225440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ce8c-3f45-4aff-a630-620a2929b183 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:05.226233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ce8c-3f45-4aff-a630-620a2929b183 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:05.226812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117ce8c-3f45-4aff-a630-620a2929b183 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:05.228220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117ce8c-3f45-4aff-a630-620a2929b183 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:05.229850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7117ce8c-3f45-4aff-a630-620a2929b183 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:05.232234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:05.243997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89095460-e35b-46d1-a893-38666a81adf7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-7117ce8c-3f45-4aff-a630-620a2929b183 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:05.247487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:05.248591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 26 02:53:05.249836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:05.252468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 483/956] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:53:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:05.262111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117ce8c-3f45-4aff-a630-620a2929b183 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:05.285443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8770d-cf68-477d-9d11-c3a67855e817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:05.286191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8770d-cf68-477d-9d11-c3a67855e817 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:05.286482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8770d-cf68-477d-9d11-c3a67855e817 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:05.286825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8770d-cf68-477d-9d11-c3a67855e817 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:05.287631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8770d-cf68-477d-9d11-c3a67855e817 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:05.287945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8770d-cf68-477d-9d11-c3a67855e817 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:05.289435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8770d-cf68-477d-9d11-c3a67855e817 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:05.304461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 474/957] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:53:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:05.309995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8770d-cf68-477d-9d11-c3a67855e817 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:05.365366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:05.384214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117ce8c-3f45-4aff-a630-620a2929b183 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:05.420018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8770d-cf68-477d-9d11-c3a67855e817 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:05.421988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96ae89-fced-4c79-ac02-94ef5bb30f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:05.422978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96ae89-fced-4c79-ac02-94ef5bb30f9f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:05.423487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96ae89-fced-4c79-ac02-94ef5bb30f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:05.423980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96ae89-fced-4c79-ac02-94ef5bb30f9f None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:05.424905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96ae89-fced-4c79-ac02-94ef5bb30f9f None admin] RBAC: Adding query filter params (name=compute) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:05.425422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96ae89-fced-4c79-ac02-94ef5bb30f9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:05.427143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96ae89-fced-4c79-ac02-94ef5bb30f9f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:05.433453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 484/958] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:53:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:05.438992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96ae89-fced-4c79-ac02-94ef5bb30f9f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:05.457081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96bb416-a02d-4f2b-bb6f-ad147ae2b801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:05.457972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bb416-a02d-4f2b-bb6f-ad147ae2b801 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:05.458296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bb416-a02d-4f2b-bb6f-ad147ae2b801 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:05.458572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96bb416-a02d-4f2b-bb6f-ad147ae2b801 None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/828fd11b79d045dc85e0973cd255bcf7/roles/d34bc405b0c04033a57dc7dae296fdee` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:05.461011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bb416-a02d-4f2b-bb6f-ad147ae2b801 None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=828fd11b79d045dc85e0973cd255bcf7, role_id=d34bc405b0c04033a57dc7dae296fdee)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:05.462491 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96bb416-a02d-4f2b-bb6f-ad147ae2b801 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:05.485044 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 475/959] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:53:05 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/828fd11b79d045dc85e0973cd255bcf7/roles/d34bc405b0c04033a57dc7dae296fdee => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 02:53:05.518713 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:05.572825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96ae89-fced-4c79-ac02-94ef5bb30f9f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:05.611534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ceb01-4ee8-4fd6-9e70-b7874e00f891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:05.612267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ceb01-4ee8-4fd6-9e70-b7874e00f891 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:05.612557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ceb01-4ee8-4fd6-9e70-b7874e00f891 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:05.612833 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ceb01-4ee8-4fd6-9e70-b7874e00f891 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:05.613564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ceb01-4ee8-4fd6-9e70-b7874e00f891 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:05.613872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ceb01-4ee8-4fd6-9e70-b7874e00f891 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:05.615338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ceb01-4ee8-4fd6-9e70-b7874e00f891 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:05.622215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 485/960] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:53:05 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:06.133726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d9419-083b-490f-b1c3-51f9d80b522a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:06.134303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d9419-083b-490f-b1c3-51f9d80b522a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:06.134303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1d9419-083b-490f-b1c3-51f9d80b522a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:06.136604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 476/961] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:06.142533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26550b05-d5ea-4131-9e76-34687b26e1fb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:06.143041 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26550b05-d5ea-4131-9e76-34687b26e1fb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:06.143141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26550b05-d5ea-4131-9e76-34687b26e1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:06.161215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-26550b05-d5ea-4131-9e76-34687b26e1fb None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:06.162441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-26550b05-d5ea-4131-9e76-34687b26e1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:06.231887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 486/962] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:06.257747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d9419-083b-490f-b1c3-51f9d80b522a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:06.364030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1d9419-083b-490f-b1c3-51f9d80b522a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:06.372954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:06.401549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139d40d-b098-4b60-8ebf-38e4c96e2ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:06.402181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139d40d-b098-4b60-8ebf-38e4c96e2ae6 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:06.402467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139d40d-b098-4b60-8ebf-38e4c96e2ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:06.402741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0139d40d-b098-4b60-8ebf-38e4c96e2ae6 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:06.403444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139d40d-b098-4b60-8ebf-38e4c96e2ae6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:06.404867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0139d40d-b098-4b60-8ebf-38e4c96e2ae6 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:06.418721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.sql.core [None req-0139d40d-b098-4b60-8ebf-38e4c96e2ae6 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 26 02:53:06.418721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 26 02:53:06.418721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [parameters: {'user_id': 'da27081a03ef4918b36b4d544e160116', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 26 02:53:06.418721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80103) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0139d40d-b098-4b60-8ebf-38e4c96e2ae6 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = self._query(query) Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application conn.query(q) Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result.read() Jun 26 02:53:06.426637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 26 02:53:06.427524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application next(self.gen) Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application yield resource Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application next(self.gen) Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.session.rollback() Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.force_reraise() Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise self.value Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application session.commit() Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 26 02:53:06.428488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._prepare_impl() Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.session.flush() Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._flush(objects) Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application compat.raise_( Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application flush_context.execute() Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rec.execute(self) Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 26 02:53:06.429386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 26 02:53:06.430343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result = self._query(query) Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application conn.query(q) Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application result.read() Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 26 02:53:06.431205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [parameters: {'user_id': 'da27081a03ef4918b36b4d544e160116', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:06.432068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 26 02:53:06.432873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 26 02:53:06.437002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 26 02:53:06.437002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 26 02:53:06.437002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 26 02:53:06.437002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 26 02:53:06.437002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 26 02:53:06.437002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:06.437002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 477/963] 10.222.0.13 () {60 vars in 1131 bytes} [Sun Jun 26 02:53:06 2022] POST /identity/v3/users => generated 160 bytes in 171 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:06.442636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26550b05-d5ea-4131-9e76-34687b26e1fb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:06.494852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:06.537441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26550b05-d5ea-4131-9e76-34687b26e1fb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:06.572917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cce7f05-3d75-41f3-abf7-6283ec1db09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:06.573723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce7f05-3d75-41f3-abf7-6283ec1db09f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:06.574006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce7f05-3d75-41f3-abf7-6283ec1db09f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:06.574321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cce7f05-3d75-41f3-abf7-6283ec1db09f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:06.581716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce7f05-3d75-41f3-abf7-6283ec1db09f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:06.583183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cce7f05-3d75-41f3-abf7-6283ec1db09f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1cce7f05-3d75-41f3-abf7-6283ec1db09f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:53:06.598341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:06.600695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:06.603080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 487/964] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:53:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:06.608576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139d40d-b098-4b60-8ebf-38e4c96e2ae6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:06.720014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0139d40d-b098-4b60-8ebf-38e4c96e2ae6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:06.757705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043f8a-82cd-4666-b3ce-63e12af9f1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:06.758810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043f8a-82cd-4666-b3ce-63e12af9f1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:06.759359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043f8a-82cd-4666-b3ce-63e12af9f1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:06.760363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043f8a-82cd-4666-b3ce-63e12af9f1b7 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:06.761407 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043f8a-82cd-4666-b3ce-63e12af9f1b7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:06.761926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043f8a-82cd-4666-b3ce-63e12af9f1b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:06.763802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043f8a-82cd-4666-b3ce-63e12af9f1b7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:06.806932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 478/965] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Jun 26 02:53:06 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:07.165111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3d534-3af9-4b58-ae45-2dce3659780e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.165647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3d534-3af9-4b58-ae45-2dce3659780e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.165647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3d534-3af9-4b58-ae45-2dce3659780e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.167389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 488/966] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:07.172509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9fe4-f8a5-4249-a744-8bd6325e292e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.172912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9fe4-f8a5-4249-a744-8bd6325e292e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.173126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb9fe4-f8a5-4249-a744-8bd6325e292e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.190488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c8bb9fe4-f8a5-4249-a744-8bd6325e292e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:07.192003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c8bb9fe4-f8a5-4249-a744-8bd6325e292e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:07.216213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:07.264268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 479/967] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:07.291457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3d534-3af9-4b58-ae45-2dce3659780e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:07.394396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3d534-3af9-4b58-ae45-2dce3659780e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:07.410639 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:07.430222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b661c-e212-44dd-afa0-b75ba60b80d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:07.430953 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b661c-e212-44dd-afa0-b75ba60b80d5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.431238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b661c-e212-44dd-afa0-b75ba60b80d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.432144 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b661c-e212-44dd-afa0-b75ba60b80d5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.432803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b661c-e212-44dd-afa0-b75ba60b80d5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:07.434210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9b661c-e212-44dd-afa0-b75ba60b80d5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-1a9b661c-e212-44dd-afa0-b75ba60b80d5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:07.446351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:07.447316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:07.448141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:07.454210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 489/968] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:53:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:07.454348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb9fe4-f8a5-4249-a744-8bd6325e292e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:07.557015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb9fe4-f8a5-4249-a744-8bd6325e292e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:07.561723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ccaad-0d2e-486f-ab37-3ad11300c1ec None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.562016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ccaad-0d2e-486f-ab37-3ad11300c1ec None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.562345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2ccaad-0d2e-486f-ab37-3ad11300c1ec None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.563878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 490/969] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:07.568724 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4907d2c-504a-445b-bce4-b8bee76d9b93 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.569019 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4907d2c-504a-445b-bce4-b8bee76d9b93 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.569305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4907d2c-504a-445b-bce4-b8bee76d9b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.586007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a4907d2c-504a-445b-bce4-b8bee76d9b93 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:07.587226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a4907d2c-504a-445b-bce4-b8bee76d9b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:07.596187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f684184-ef69-497a-8e7f-6936506cb208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:07.597075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f684184-ef69-497a-8e7f-6936506cb208 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.597716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f684184-ef69-497a-8e7f-6936506cb208 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.598000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f684184-ef69-497a-8e7f-6936506cb208 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.598777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f684184-ef69-497a-8e7f-6936506cb208 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:07.599213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f684184-ef69-497a-8e7f-6936506cb208 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:07.600600 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f684184-ef69-497a-8e7f-6936506cb208 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:07.611404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 480/970] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Jun 26 02:53:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:07.612377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f684184-ef69-497a-8e7f-6936506cb208 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:07.637287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:07.663587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 491/971] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:07.679762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa69a6-d29a-47ac-9d13-2576baafeb92 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.680060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa69a6-d29a-47ac-9d13-2576baafeb92 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.680380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaa69a6-d29a-47ac-9d13-2576baafeb92 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.682745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 492/972] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:07.687813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77449233-6e04-4373-82e3-10aec9d734c5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.688108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77449233-6e04-4373-82e3-10aec9d734c5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.688386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77449233-6e04-4373-82e3-10aec9d734c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.705097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-77449233-6e04-4373-82e3-10aec9d734c5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:07.706305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-77449233-6e04-4373-82e3-10aec9d734c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:07.715853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f684184-ef69-497a-8e7f-6936506cb208 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:07.749753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf46a5f-f271-4b4d-943b-a0fe84191e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:07.750454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf46a5f-f271-4b4d-943b-a0fe84191e0a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.750742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf46a5f-f271-4b4d-943b-a0fe84191e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.751028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf46a5f-f271-4b4d-943b-a0fe84191e0a None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.751738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf46a5f-f271-4b4d-943b-a0fe84191e0a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:07.752056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf46a5f-f271-4b4d-943b-a0fe84191e0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:07.753415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf46a5f-f271-4b4d-943b-a0fe84191e0a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:07.758276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 481/973] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Jun 26 02:53:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:07.761598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf46a5f-f271-4b4d-943b-a0fe84191e0a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:07.771869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 493/974] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:07.772607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77449233-6e04-4373-82e3-10aec9d734c5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:07.814426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:07.846743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:07.863640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf46a5f-f271-4b4d-943b-a0fe84191e0a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:07.879424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77449233-6e04-4373-82e3-10aec9d734c5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:07.902025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1b53b-4244-4584-9cd1-de80a4f9fee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:07.902861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1b53b-4244-4584-9cd1-de80a4f9fee6 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.903146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1b53b-4244-4584-9cd1-de80a4f9fee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.903423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1b53b-4244-4584-9cd1-de80a4f9fee6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.912834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e3b7f-d244-422e-861f-b285e94ca4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:07.913411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d1b53b-4244-4584-9cd1-de80a4f9fee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:07.913520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e3b7f-d244-422e-861f-b285e94ca4cc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:07.913756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e3b7f-d244-422e-861f-b285e94ca4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:07.914033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e3b7f-d244-422e-861f-b285e94ca4cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:07.914857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e3b7f-d244-422e-861f-b285e94ca4cc None admin] RBAC: Adding query filter params (interface=admin, service_id=1592033d11d043eaa54ea9e8460f9918, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:07.915173 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e3b7f-d244-422e-861f-b285e94ca4cc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:07.916588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e3b7f-d244-422e-861f-b285e94ca4cc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-61d1b53b-4244-4584-9cd1-de80a4f9fee6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:07.921511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:07.922770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 482/975] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Jun 26 02:53:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:07.923797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1b53b-4244-4584-9cd1-de80a4f9fee6 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:07.926656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 494/976] 10.222.0.13 () {56 vars in 1249 bytes} [Sun Jun 26 02:53:07 2022] GET /identity/v3/endpoints?service_id=1592033d11d043eaa54ea9e8460f9918&interface=admin®ion_id=RegionOne => generated 193 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:07.928518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e3b7f-d244-422e-861f-b285e94ca4cc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:08.026350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1b53b-4244-4584-9cd1-de80a4f9fee6 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:08.028666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e3b7f-d244-422e-861f-b285e94ca4cc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:08.031725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:08.060246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f01cf-a0a9-414b-9390-85deee730b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:08.060909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f01cf-a0a9-414b-9390-85deee730b6f None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:08.061197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f01cf-a0a9-414b-9390-85deee730b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:08.061478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f01cf-a0a9-414b-9390-85deee730b6f None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:08.062235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f01cf-a0a9-414b-9390-85deee730b6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:08.063604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f01cf-a0a9-414b-9390-85deee730b6f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:08.064063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9736ec-cb25-485d-8692-b7818b36999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:08.064725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9736ec-cb25-485d-8692-b7818b36999c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:08.065014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9736ec-cb25-485d-8692-b7818b36999c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:08.065292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9736ec-cb25-485d-8692-b7818b36999c None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:08.066051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9736ec-cb25-485d-8692-b7818b36999c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:08.066378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9736ec-cb25-485d-8692-b7818b36999c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:08.067764 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9736ec-cb25-485d-8692-b7818b36999c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:08.078103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 495/978] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Jun 26 02:53:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:08.078262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 483/978] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:53:07 2022] POST /identity/v3/services => generated 242 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:53:08.078354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9736ec-cb25-485d-8692-b7818b36999c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:08.083538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f01cf-a0a9-414b-9390-85deee730b6f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:08.096196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:08.201633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9736ec-cb25-485d-8692-b7818b36999c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:08.209027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f01cf-a0a9-414b-9390-85deee730b6f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:08.243005 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b98548-8210-40fa-bc7b-2e12db7e88cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:08.244714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b98548-8210-40fa-bc7b-2e12db7e88cc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:08.245001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b98548-8210-40fa-bc7b-2e12db7e88cc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:08.245278 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b98548-8210-40fa-bc7b-2e12db7e88cc None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:08.245966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b98548-8210-40fa-bc7b-2e12db7e88cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:08.247418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b98548-8210-40fa-bc7b-2e12db7e88cc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:08.249425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711c946-ce4e-41cd-aedf-9eabb1477eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:08.250426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711c946-ce4e-41cd-aedf-9eabb1477eef None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:08.250896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711c946-ce4e-41cd-aedf-9eabb1477eef None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:08.251175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711c946-ce4e-41cd-aedf-9eabb1477eef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:08.256628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 496/979] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Jun 26 02:53:08 2022] GET /identity/v3/services => generated 1727 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 02:53:08.262232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711c946-ce4e-41cd-aedf-9eabb1477eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:08.264182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7711c946-ce4e-41cd-aedf-9eabb1477eef None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-7711c946-ce4e-41cd-aedf-9eabb1477eef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:08.283462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:08.284416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:08.286472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:08.286472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:08.286472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:08.286472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:08.286472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 484/980] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:08.292398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b98548-8210-40fa-bc7b-2e12db7e88cc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:08.327524 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:08.404137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b98548-8210-40fa-bc7b-2e12db7e88cc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:08.441314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be333e1a-0b53-4439-97aa-bb64df7f24f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:08.441991 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be333e1a-0b53-4439-97aa-bb64df7f24f4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:08.442303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be333e1a-0b53-4439-97aa-bb64df7f24f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:08.442582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be333e1a-0b53-4439-97aa-bb64df7f24f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:08.443315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be333e1a-0b53-4439-97aa-bb64df7f24f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:08.443627 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be333e1a-0b53-4439-97aa-bb64df7f24f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:08.444993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be333e1a-0b53-4439-97aa-bb64df7f24f4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:08.458291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 497/981] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:08.464107 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711c946-ce4e-41cd-aedf-9eabb1477eef None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:08.521745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:08.568734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7711c946-ce4e-41cd-aedf-9eabb1477eef None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:08.604372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72b7d7-ead8-444b-87e1-8ad05855aafa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:08.605471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72b7d7-ead8-444b-87e1-8ad05855aafa None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:08.605967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72b7d7-ead8-444b-87e1-8ad05855aafa None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:08.606434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb72b7d7-ead8-444b-87e1-8ad05855aafa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:08.614385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72b7d7-ead8-444b-87e1-8ad05855aafa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:08.616024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb72b7d7-ead8-444b-87e1-8ad05855aafa None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-cb72b7d7-ead8-444b-87e1-8ad05855aafa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:53:08.631966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:08.633002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:08.633957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 485/982] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Jun 26 02:53:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:08.641509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be333e1a-0b53-4439-97aa-bb64df7f24f4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:08.735734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:08.747002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be333e1a-0b53-4439-97aa-bb64df7f24f4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:08.781609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6c6b-720c-44fd-9c5b-f72e1dc078ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:08.782325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6c6b-720c-44fd-9c5b-f72e1dc078ff None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:08.782612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6c6b-720c-44fd-9c5b-f72e1dc078ff None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:08.782888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f6c6b-720c-44fd-9c5b-f72e1dc078ff None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:08.783723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f6c6b-720c-44fd-9c5b-f72e1dc078ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:08.784035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f6c6b-720c-44fd-9c5b-f72e1dc078ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:08.785463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f6c6b-720c-44fd-9c5b-f72e1dc078ff None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:08.809146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 498/983] 10.222.0.13 () {56 vars in 1149 bytes} [Sun Jun 26 02:53:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:08.810526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72b7d7-ead8-444b-87e1-8ad05855aafa None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:08.915034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb72b7d7-ead8-444b-87e1-8ad05855aafa None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:08.929537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:08.950824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7125d2-7e15-416d-b3f2-ec495470b8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:08.951479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7125d2-7e15-416d-b3f2-ec495470b8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:08.951764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7125d2-7e15-416d-b3f2-ec495470b8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:08.952045 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7125d2-7e15-416d-b3f2-ec495470b8d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:09.019404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7125d2-7e15-416d-b3f2-ec495470b8d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:09.020858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7125d2-7e15-416d-b3f2-ec495470b8d9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:09.122099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 486/984] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:08 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:53:09.123477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6c6b-720c-44fd-9c5b-f72e1dc078ff None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:09.226166 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f6c6b-720c-44fd-9c5b-f72e1dc078ff None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:09.260782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41bd15-2b07-4ca3-82b7-0492540a3824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:09.261575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41bd15-2b07-4ca3-82b7-0492540a3824 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:09.261857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41bd15-2b07-4ca3-82b7-0492540a3824 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:09.262169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f41bd15-2b07-4ca3-82b7-0492540a3824 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:09.272160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41bd15-2b07-4ca3-82b7-0492540a3824 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:09.273592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f41bd15-2b07-4ca3-82b7-0492540a3824 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:09.283978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-8f41bd15-2b07-4ca3-82b7-0492540a3824 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:09.295870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:09.296737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:09.297625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:09.297625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:09.297625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:09.297625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:09.298425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 499/985] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:09.302701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7125d2-7e15-416d-b3f2-ec495470b8d9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:09.318325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:09.405716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7125d2-7e15-416d-b3f2-ec495470b8d9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:09.439000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e3eb0-3c27-4a2c-9d94-77ef52f95e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:09.439660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e3eb0-3c27-4a2c-9d94-77ef52f95e55 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:09.439947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e3eb0-3c27-4a2c-9d94-77ef52f95e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:09.440224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e3eb0-3c27-4a2c-9d94-77ef52f95e55 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:09.440939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e3eb0-3c27-4a2c-9d94-77ef52f95e55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:09.441246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e3eb0-3c27-4a2c-9d94-77ef52f95e55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:09.442623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044e3eb0-3c27-4a2c-9d94-77ef52f95e55 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:09.458883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 487/986] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:09.460213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41bd15-2b07-4ca3-82b7-0492540a3824 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:09.559778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f41bd15-2b07-4ca3-82b7-0492540a3824 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:09.593168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53887aa2-929a-4139-8fcb-ffeaca39b03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:09.593956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53887aa2-929a-4139-8fcb-ffeaca39b03e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:09.594265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53887aa2-929a-4139-8fcb-ffeaca39b03e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:09.594574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53887aa2-929a-4139-8fcb-ffeaca39b03e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:09.601534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53887aa2-929a-4139-8fcb-ffeaca39b03e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:09.603115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53887aa2-929a-4139-8fcb-ffeaca39b03e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-53887aa2-929a-4139-8fcb-ffeaca39b03e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:53:09.617157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:09.618268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:09.626323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:53:09.626323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:53:09.626323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:53:09.626323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:53:09.626323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:53:09.626323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:53:09.626323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:09.626323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:09.626323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 500/987] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:53:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:09.626851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e3eb0-3c27-4a2c-9d94-77ef52f95e55 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:09.703581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:09.730149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e3eb0-3c27-4a2c-9d94-77ef52f95e55 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:09.763615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc26dc30-af8d-4f96-b2f5-3c3be3353efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:09.764270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26dc30-af8d-4f96-b2f5-3c3be3353efc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:09.764554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26dc30-af8d-4f96-b2f5-3c3be3353efc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:09.764836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc26dc30-af8d-4f96-b2f5-3c3be3353efc None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:09.765617 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26dc30-af8d-4f96-b2f5-3c3be3353efc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:09.765927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26dc30-af8d-4f96-b2f5-3c3be3353efc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:09.767375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc26dc30-af8d-4f96-b2f5-3c3be3353efc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:09.784107 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 488/988] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:53:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:09.785356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53887aa2-929a-4139-8fcb-ffeaca39b03e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:09.844769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:09.893411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53887aa2-929a-4139-8fcb-ffeaca39b03e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:09.927522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757e3545-ab52-4c14-aec2-d474bf82ece8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:09.928178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e3545-ab52-4c14-aec2-d474bf82ece8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:09.928461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e3545-ab52-4c14-aec2-d474bf82ece8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:09.928737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e3545-ab52-4c14-aec2-d474bf82ece8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:09.929566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e3545-ab52-4c14-aec2-d474bf82ece8 None admin] RBAC: Adding query filter params (role.id=d34bc405b0c04033a57dc7dae296fdee, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=828fd11b79d045dc85e0973cd255bcf7) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:09.929880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e3545-ab52-4c14-aec2-d474bf82ece8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:09.931364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757e3545-ab52-4c14-aec2-d474bf82ece8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:09.946094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 501/989] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:53:09 2022] GET /identity/v3/role_assignments?user.id=828fd11b79d045dc85e0973cd255bcf7&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=d34bc405b0c04033a57dc7dae296fdee => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:10.137314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab633d8b-19a9-4096-b22c-5388298c6d9e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.137711 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab633d8b-19a9-4096-b22c-5388298c6d9e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.137878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab633d8b-19a9-4096-b22c-5388298c6d9e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.139471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 489/990] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:10.144532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8725e-bcaa-40da-ae16-ed81f8ef1433 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.144831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8725e-bcaa-40da-ae16-ed81f8ef1433 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.145116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff8725e-bcaa-40da-ae16-ed81f8ef1433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.162301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cff8725e-bcaa-40da-ae16-ed81f8ef1433 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:10.163501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cff8725e-bcaa-40da-ae16-ed81f8ef1433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:10.232710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 502/991] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:10.255708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab633d8b-19a9-4096-b22c-5388298c6d9e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:10.301573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406806e-ce49-4ec5-9580-796cd52271dc None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.301913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406806e-ce49-4ec5-9580-796cd52271dc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.302164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406806e-ce49-4ec5-9580-796cd52271dc None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.303679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 503/992] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:10.308368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99858be5-5cd9-4728-8692-5f9fa7a8e7b1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.308653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99858be5-5cd9-4728-8692-5f9fa7a8e7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.308927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99858be5-5cd9-4728-8692-5f9fa7a8e7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.325674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-99858be5-5cd9-4728-8692-5f9fa7a8e7b1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:10.326884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-99858be5-5cd9-4728-8692-5f9fa7a8e7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:10.364305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab633d8b-19a9-4096-b22c-5388298c6d9e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:10.380506 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:10.399069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f6ebc-e532-4d09-8e8d-2c5a6c16221a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:10.399788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f6ebc-e532-4d09-8e8d-2c5a6c16221a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.400089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f6ebc-e532-4d09-8e8d-2c5a6c16221a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.400897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f6ebc-e532-4d09-8e8d-2c5a6c16221a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.402101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f6ebc-e532-4d09-8e8d-2c5a6c16221a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:10.402246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 504/993] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:10.403005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f6ebc-e532-4d09-8e8d-2c5a6c16221a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:10.404801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ca8f6ebc-e532-4d09-8e8d-2c5a6c16221a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:10.417214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:10.418306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:10.419124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 490/994] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:53:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:10.425465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f6ebc-e532-4d09-8e8d-2c5a6c16221a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:10.425753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99858be5-5cd9-4728-8692-5f9fa7a8e7b1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:10.527806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f6ebc-e532-4d09-8e8d-2c5a6c16221a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:10.539810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99858be5-5cd9-4728-8692-5f9fa7a8e7b1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:10.563986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec4ad5-4fb5-464a-a6f4-6f36d1015857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:10.565028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec4ad5-4fb5-464a-a6f4-6f36d1015857 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.565313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec4ad5-4fb5-464a-a6f4-6f36d1015857 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.565587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec4ad5-4fb5-464a-a6f4-6f36d1015857 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.566276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec4ad5-4fb5-464a-a6f4-6f36d1015857 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:10.567633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ec4ad5-4fb5-464a-a6f4-6f36d1015857 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:10.575148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad83c3-7dab-4e91-869c-ae1d2f531794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:10.575808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad83c3-7dab-4e91-869c-ae1d2f531794 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.576089 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad83c3-7dab-4e91-869c-ae1d2f531794 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.576365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ad83c3-7dab-4e91-869c-ae1d2f531794 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.577065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad83c3-7dab-4e91-869c-ae1d2f531794 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:10.577371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad83c3-7dab-4e91-869c-ae1d2f531794 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:10.578746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ad83c3-7dab-4e91-869c-ae1d2f531794 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e6ec4ad5-4fb5-464a-a6f4-6f36d1015857 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:10.580104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:10.581031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:10.581842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 491/995] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Jun 26 02:53:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:10.591358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 505/996] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:53:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:10.591511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec4ad5-4fb5-464a-a6f4-6f36d1015857 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:10.592167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad83c3-7dab-4e91-869c-ae1d2f531794 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:10.641946 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:10.659970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:10.696713 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ad83c3-7dab-4e91-869c-ae1d2f531794 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:10.702982 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ec4ad5-4fb5-464a-a6f4-6f36d1015857 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:10.732636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4581957-b3a4-4f7b-aa3e-e4786fade5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:10.733302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4581957-b3a4-4f7b-aa3e-e4786fade5ed None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.733588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4581957-b3a4-4f7b-aa3e-e4786fade5ed None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.733859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4581957-b3a4-4f7b-aa3e-e4786fade5ed None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.734654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4581957-b3a4-4f7b-aa3e-e4786fade5ed None admin] RBAC: Adding query filter params (type=compute) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:10.734966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4581957-b3a4-4f7b-aa3e-e4786fade5ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:10.736354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4581957-b3a4-4f7b-aa3e-e4786fade5ed None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:10.738133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bafc65-107b-4793-98f0-5efa5ae8da92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:10.738789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bafc65-107b-4793-98f0-5efa5ae8da92 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.739071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bafc65-107b-4793-98f0-5efa5ae8da92 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.739341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bafc65-107b-4793-98f0-5efa5ae8da92 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.740034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bafc65-107b-4793-98f0-5efa5ae8da92 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:10.740348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bafc65-107b-4793-98f0-5efa5ae8da92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:10.746902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 506/997] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:53:10 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:10.747033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bafc65-107b-4793-98f0-5efa5ae8da92 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:10.747716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4581957-b3a4-4f7b-aa3e-e4786fade5ed None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:10.751705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 492/998] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Jun 26 02:53:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:10.757054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bafc65-107b-4793-98f0-5efa5ae8da92 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:10.859205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4581957-b3a4-4f7b-aa3e-e4786fade5ed None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:10.866074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bafc65-107b-4793-98f0-5efa5ae8da92 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:10.874875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:10.893335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:10.899440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c21e3d-c5b0-42e7-8deb-7a89536565a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:10.900165 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c21e3d-c5b0-42e7-8deb-7a89536565a3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.900412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c21e3d-c5b0-42e7-8deb-7a89536565a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.900690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c21e3d-c5b0-42e7-8deb-7a89536565a3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.901478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c21e3d-c5b0-42e7-8deb-7a89536565a3 None admin] RBAC: Adding query filter params (interface=public, service_id=9158c623705847f6b44e3d694b27bd8a, region_id=RegionOne) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:10.901793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c21e3d-c5b0-42e7-8deb-7a89536565a3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:10.903162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398695a-3913-49c6-8863-ef8329baf4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:10.903295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c21e3d-c5b0-42e7-8deb-7a89536565a3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:10.903819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398695a-3913-49c6-8863-ef8329baf4f5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:10.904101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398695a-3913-49c6-8863-ef8329baf4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:10.904379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398695a-3913-49c6-8863-ef8329baf4f5 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:10.905078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398695a-3913-49c6-8863-ef8329baf4f5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:10.905906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398695a-3913-49c6-8863-ef8329baf4f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:10.907027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398695a-3913-49c6-8863-ef8329baf4f5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:10.909185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 507/999] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Jun 26 02:53:10 2022] GET /identity/v3/endpoints?service_id=9158c623705847f6b44e3d694b27bd8a&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:10.913315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 493/1000] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Jun 26 02:53:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:10.918292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398695a-3913-49c6-8863-ef8329baf4f5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:10.918422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c21e3d-c5b0-42e7-8deb-7a89536565a3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:11.020506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398695a-3913-49c6-8863-ef8329baf4f5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:11.026835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c21e3d-c5b0-42e7-8deb-7a89536565a3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:11.054223 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9633e-da2b-40c5-adcf-767ffcede21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:11.054883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9633e-da2b-40c5-adcf-767ffcede21a None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:11.055166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9633e-da2b-40c5-adcf-767ffcede21a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:11.055442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9633e-da2b-40c5-adcf-767ffcede21a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:11.056197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9633e-da2b-40c5-adcf-767ffcede21a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:11.058247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9633e-da2b-40c5-adcf-767ffcede21a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:11.066520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b4734-1aef-4dc1-8de7-bc347052069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:11.067571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b4734-1aef-4dc1-8de7-bc347052069f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:11.068205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b4734-1aef-4dc1-8de7-bc347052069f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:11.068685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b4734-1aef-4dc1-8de7-bc347052069f None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:11.069613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b4734-1aef-4dc1-8de7-bc347052069f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:11.071248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b4734-1aef-4dc1-8de7-bc347052069f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:11.074320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 494/1001] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Jun 26 02:53:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:53:11.086577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 508/1002] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Jun 26 02:53:10 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 02:53:12.009609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75b84-6ae7-40b0-9027-b67d613f3614 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:12.010328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75b84-6ae7-40b0-9027-b67d613f3614 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:12.010328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c75b84-6ae7-40b0-9027-b67d613f3614 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:12.011659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 495/1003] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:12.016805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f977cd1-24ed-482a-b0d7-3af7f8119e90 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:12.017217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f977cd1-24ed-482a-b0d7-3af7f8119e90 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:12.017392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f977cd1-24ed-482a-b0d7-3af7f8119e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:12.033771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9f977cd1-24ed-482a-b0d7-3af7f8119e90 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:12.034979 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9f977cd1-24ed-482a-b0d7-3af7f8119e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:12.070577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:12.109466 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 509/1004] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:12 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:12.131870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75b84-6ae7-40b0-9027-b67d613f3614 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:12.184121 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:12.241282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c75b84-6ae7-40b0-9027-b67d613f3614 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:12.275537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3253ec4-12af-4a4e-80fd-c585db458a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:12.276323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3253ec4-12af-4a4e-80fd-c585db458a3c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:12.276607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3253ec4-12af-4a4e-80fd-c585db458a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:12.276881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3253ec4-12af-4a4e-80fd-c585db458a3c None admin] PATH_INFO: `/v3/services/image` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:12.277532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3253ec4-12af-4a4e-80fd-c585db458a3c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:12.278919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3253ec4-12af-4a4e-80fd-c585db458a3c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-c3253ec4-12af-4a4e-80fd-c585db458a3c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:12.291991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:12.293373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:12.294556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 496/1005] 10.222.0.13 () {56 vars in 1105 bytes} [Sun Jun 26 02:53:12 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:12.297626 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f977cd1-24ed-482a-b0d7-3af7f8119e90 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:12.396101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f977cd1-24ed-482a-b0d7-3af7f8119e90 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:12.428258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed6a420-8a7f-40e4-84c6-2b2b6114f7a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:12.428906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6a420-8a7f-40e4-84c6-2b2b6114f7a1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:12.429187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6a420-8a7f-40e4-84c6-2b2b6114f7a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:12.429461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed6a420-8a7f-40e4-84c6-2b2b6114f7a1 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:12.430183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6a420-8a7f-40e4-84c6-2b2b6114f7a1 None admin] RBAC: Adding query filter params (name=image) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:12.430495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6a420-8a7f-40e4-84c6-2b2b6114f7a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:12.432352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed6a420-8a7f-40e4-84c6-2b2b6114f7a1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:12.437204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 510/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:12.442579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3253ec4-12af-4a4e-80fd-c585db458a3c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:12.537134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3253ec4-12af-4a4e-80fd-c585db458a3c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:12.550744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:12.571883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974ef3c-e3d3-4e7f-93ba-157538a89070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:12.572572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974ef3c-e3d3-4e7f-93ba-157538a89070 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:12.572852 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974ef3c-e3d3-4e7f-93ba-157538a89070 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:12.573125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974ef3c-e3d3-4e7f-93ba-157538a89070 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:12.573824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974ef3c-e3d3-4e7f-93ba-157538a89070 None admin] RBAC: Adding query filter params (type=image) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:12.574155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974ef3c-e3d3-4e7f-93ba-157538a89070 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:12.575507 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974ef3c-e3d3-4e7f-93ba-157538a89070 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:12.586090 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 497/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:12 2022] GET /identity/v3/services?type=image => generated 122 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:13.126522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35d627-c5e3-46ca-843a-e8bcfc224f4a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:13.127043 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35d627-c5e3-46ca-843a-e8bcfc224f4a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:13.127124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f35d627-c5e3-46ca-843a-e8bcfc224f4a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:13.128603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 511/1008] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:13.133482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b769ad-81cb-487a-8cfa-ca957e05a9e3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:13.133855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b769ad-81cb-487a-8cfa-ca957e05a9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:13.134057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b769ad-81cb-487a-8cfa-ca957e05a9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:13.150905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-30b769ad-81cb-487a-8cfa-ca957e05a9e3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:13.152078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-30b769ad-81cb-487a-8cfa-ca957e05a9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:13.191109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0d634-ab84-4a6a-86d0-30cbe8ea48d9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:13.191391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0d634-ab84-4a6a-86d0-30cbe8ea48d9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:13.191667 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca0d634-ab84-4a6a-86d0-30cbe8ea48d9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:13.193194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 512/1009] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:13.198089 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac785a-027e-4324-8f43-2ffbd963e49e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:13.198384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac785a-027e-4324-8f43-2ffbd963e49e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:13.198654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac785a-027e-4324-8f43-2ffbd963e49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:13.215340 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c1ac785a-027e-4324-8f43-2ffbd963e49e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:13.216513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c1ac785a-027e-4324-8f43-2ffbd963e49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:13.219922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 498/1010] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:13 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:13.241737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:13.242390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b769ad-81cb-487a-8cfa-ca957e05a9e3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:13.288030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 513/1011] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:13 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:13.312417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac785a-027e-4324-8f43-2ffbd963e49e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:13.345246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b769ad-81cb-487a-8cfa-ca957e05a9e3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:13.355202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:13.382107 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c89a65-ff7a-4b71-82cb-c28d9ecfb3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:13.382785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c89a65-ff7a-4b71-82cb-c28d9ecfb3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:13.383067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c89a65-ff7a-4b71-82cb-c28d9ecfb3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:13.383350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c89a65-ff7a-4b71-82cb-c28d9ecfb3c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:13.385051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c89a65-ff7a-4b71-82cb-c28d9ecfb3c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:13.386496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c89a65-ff7a-4b71-82cb-c28d9ecfb3c5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:13.388151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 499/1012] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:13.393615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c89a65-ff7a-4b71-82cb-c28d9ecfb3c5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:13.422909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac785a-027e-4324-8f43-2ffbd963e49e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:13.456289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2293b-41a6-4cba-97ce-2258884e4312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:13.457085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2293b-41a6-4cba-97ce-2258884e4312 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:13.457373 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2293b-41a6-4cba-97ce-2258884e4312 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:13.457651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f2293b-41a6-4cba-97ce-2258884e4312 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:13.458332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2293b-41a6-4cba-97ce-2258884e4312 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:13.459683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f2293b-41a6-4cba-97ce-2258884e4312 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-51f2293b-41a6-4cba-97ce-2258884e4312 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:13.472187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:13.473159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:13.474234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 514/1013] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:53:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:13.479101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2293b-41a6-4cba-97ce-2258884e4312 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:13.504589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:13.507286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c89a65-ff7a-4b71-82cb-c28d9ecfb3c5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:13.543357 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05e514-9892-40d8-b69c-68fcb248d7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:13.544025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05e514-9892-40d8-b69c-68fcb248d7ce None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:13.544311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05e514-9892-40d8-b69c-68fcb248d7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:13.544588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b05e514-9892-40d8-b69c-68fcb248d7ce None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:13.545298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05e514-9892-40d8-b69c-68fcb248d7ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:13.546775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b05e514-9892-40d8-b69c-68fcb248d7ce None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:13.572766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 500/1014] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:53:13 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:53:13.602051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f2293b-41a6-4cba-97ce-2258884e4312 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:13.641445 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2fbce-4f17-4d1e-956e-54a0e0ce561c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:13.642088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2fbce-4f17-4d1e-956e-54a0e0ce561c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:13.642368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2fbce-4f17-4d1e-956e-54a0e0ce561c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:13.642646 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a2fbce-4f17-4d1e-956e-54a0e0ce561c None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:13.643372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2fbce-4f17-4d1e-956e-54a0e0ce561c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:13.643691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2fbce-4f17-4d1e-956e-54a0e0ce561c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:13.645140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a2fbce-4f17-4d1e-956e-54a0e0ce561c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:13.652410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 515/1015] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:53:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:13.657744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05e514-9892-40d8-b69c-68fcb248d7ce None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:13.690564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:13.771355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b05e514-9892-40d8-b69c-68fcb248d7ce None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:13.804794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ae04e0-6cef-44a5-aee3-e680fb9316af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:13.805516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae04e0-6cef-44a5-aee3-e680fb9316af None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:13.805805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae04e0-6cef-44a5-aee3-e680fb9316af None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:13.806142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ae04e0-6cef-44a5-aee3-e680fb9316af None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:13.807380 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae04e0-6cef-44a5-aee3-e680fb9316af None admin] RBAC: Adding query filter params (type=compute) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:13.807688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae04e0-6cef-44a5-aee3-e680fb9316af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:13.809047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ae04e0-6cef-44a5-aee3-e680fb9316af None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:13.818106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 501/1016] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Jun 26 02:53:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:13.821165 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2fbce-4f17-4d1e-956e-54a0e0ce561c None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:13.918721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:13.932302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a2fbce-4f17-4d1e-956e-54a0e0ce561c None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:13.965716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090914a2-0326-47a8-ae67-34732968ef67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:13.966455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090914a2-0326-47a8-ae67-34732968ef67 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:13.966746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090914a2-0326-47a8-ae67-34732968ef67 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:13.967023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090914a2-0326-47a8-ae67-34732968ef67 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:13.967767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090914a2-0326-47a8-ae67-34732968ef67 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:13.969127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090914a2-0326-47a8-ae67-34732968ef67 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:13.989012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 516/1017] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Jun 26 02:53:13 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:53:14.607916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5735ceb-c54a-4327-bf8f-ee696357b9e6 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:14.608430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5735ceb-c54a-4327-bf8f-ee696357b9e6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:14.608509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5735ceb-c54a-4327-bf8f-ee696357b9e6 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:14.610019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 502/1018] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:14.614939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876a217-de73-4f93-8f72-4735e2a2090f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:14.615315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876a217-de73-4f93-8f72-4735e2a2090f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:14.615500 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e876a217-de73-4f93-8f72-4735e2a2090f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:14.632971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e876a217-de73-4f93-8f72-4735e2a2090f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:14.634158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e876a217-de73-4f93-8f72-4735e2a2090f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:14.706559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 517/1019] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:14 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:14.729208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5735ceb-c54a-4327-bf8f-ee696357b9e6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:14.821138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:14.831570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5735ceb-c54a-4327-bf8f-ee696357b9e6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:14.874275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b413f32-6416-4f14-b26a-280678cb3048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:14.874941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b413f32-6416-4f14-b26a-280678cb3048 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:14.875224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b413f32-6416-4f14-b26a-280678cb3048 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:14.875496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b413f32-6416-4f14-b26a-280678cb3048 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:14.876205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b413f32-6416-4f14-b26a-280678cb3048 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:14.877556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b413f32-6416-4f14-b26a-280678cb3048 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:14.898085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 503/1020] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:53:14 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:53:15.639213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98b1fb-7513-44be-8ebf-67f675fbc851 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:15.639729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98b1fb-7513-44be-8ebf-67f675fbc851 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:15.639806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98b1fb-7513-44be-8ebf-67f675fbc851 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:15.641286 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 518/1021] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:15.646360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4e8f-4f40-4cb5-aa0d-6103d26d5dd1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:15.646772 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4e8f-4f40-4cb5-aa0d-6103d26d5dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:15.646934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed4e8f-4f40-4cb5-aa0d-6103d26d5dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:15.663134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-61ed4e8f-4f40-4cb5-aa0d-6103d26d5dd1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:15.664310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-61ed4e8f-4f40-4cb5-aa0d-6103d26d5dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:15.734622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 504/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:15.758916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98b1fb-7513-44be-8ebf-67f675fbc851 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:15.828657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:15.866136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98b1fb-7513-44be-8ebf-67f675fbc851 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:15.903796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fd87d9-78cb-45d1-8d5b-150d631cbe8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:15.904515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd87d9-78cb-45d1-8d5b-150d631cbe8c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:15.904794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd87d9-78cb-45d1-8d5b-150d631cbe8c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:15.905058 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fd87d9-78cb-45d1-8d5b-150d631cbe8c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:15.906312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd87d9-78cb-45d1-8d5b-150d631cbe8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:15.907736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fd87d9-78cb-45d1-8d5b-150d631cbe8c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:15.913830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 519/1023] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:15.915110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4e8f-4f40-4cb5-aa0d-6103d26d5dd1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:15.993512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:16.017226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed4e8f-4f40-4cb5-aa0d-6103d26d5dd1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:16.054261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4417e65-d5f2-4b40-8bc0-ab5416ba3a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:16.054931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4417e65-d5f2-4b40-8bc0-ab5416ba3a8c None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:16.055218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4417e65-d5f2-4b40-8bc0-ab5416ba3a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:16.055492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4417e65-d5f2-4b40-8bc0-ab5416ba3a8c None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:16.056224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4417e65-d5f2-4b40-8bc0-ab5416ba3a8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:16.057657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4417e65-d5f2-4b40-8bc0-ab5416ba3a8c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:16.078949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 505/1024] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:53:15 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:53:16.094419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b717a-16f7-440c-9bba-58aba7aba7d9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:16.094701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b717a-16f7-440c-9bba-58aba7aba7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:16.095203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61b717a-16f7-440c-9bba-58aba7aba7d9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:16.097024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 520/1025] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:16.103713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17b86a-3157-4e4c-831a-4990246d1b52 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:16.104236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17b86a-3157-4e4c-831a-4990246d1b52 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:16.104696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17b86a-3157-4e4c-831a-4990246d1b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:16.122428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6e17b86a-3157-4e4c-831a-4990246d1b52 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:16.124213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6e17b86a-3157-4e4c-831a-4990246d1b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:16.202167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 506/1026] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:16.223867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b717a-16f7-440c-9bba-58aba7aba7d9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:16.332155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61b717a-16f7-440c-9bba-58aba7aba7d9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:16.367573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558c715-a97c-4764-8d2e-1e6d07706217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:16.368384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558c715-a97c-4764-8d2e-1e6d07706217 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:16.368687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558c715-a97c-4764-8d2e-1e6d07706217 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:16.368968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558c715-a97c-4764-8d2e-1e6d07706217 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:16.376263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:16.380869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558c715-a97c-4764-8d2e-1e6d07706217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2558c715-a97c-4764-8d2e-1e6d07706217 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:16.389884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:16.391023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:16.391856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 521/1027] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Jun 26 02:53:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:16.397067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17b86a-3157-4e4c-831a-4990246d1b52 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:16.459649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:16.499337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17b86a-3157-4e4c-831a-4990246d1b52 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:16.533674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc596b-1ee1-4506-b2fe-acc53174630f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:16.534375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc596b-1ee1-4506-b2fe-acc53174630f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:16.534660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc596b-1ee1-4506-b2fe-acc53174630f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:16.534934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affc596b-1ee1-4506-b2fe-acc53174630f None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:16.535679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc596b-1ee1-4506-b2fe-acc53174630f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:16.535983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc596b-1ee1-4506-b2fe-acc53174630f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:16.537339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affc596b-1ee1-4506-b2fe-acc53174630f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:16.550229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 507/1028] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Jun 26 02:53:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:16.551672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558c715-a97c-4764-8d2e-1e6d07706217 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:16.656976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558c715-a97c-4764-8d2e-1e6d07706217 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:16.691081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8a662-7a19-4489-bfc9-a6d16e5d0965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:16.691876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8a662-7a19-4489-bfc9-a6d16e5d0965 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:16.692158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8a662-7a19-4489-bfc9-a6d16e5d0965 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:16.692434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c8a662-7a19-4489-bfc9-a6d16e5d0965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:16.703961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8a662-7a19-4489-bfc9-a6d16e5d0965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:16.705425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c8a662-7a19-4489-bfc9-a6d16e5d0965 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-73c8a662-7a19-4489-bfc9-a6d16e5d0965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:16.723161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:16.724071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:16.729672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:16.729672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:16.729672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:16.729672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:16.729672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 522/1029] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:16.731628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc596b-1ee1-4506-b2fe-acc53174630f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:16.834547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affc596b-1ee1-4506-b2fe-acc53174630f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:16.863376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:16.877916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f09486e-770a-4467-a18e-507c4e4d7754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:16.878616 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09486e-770a-4467-a18e-507c4e4d7754 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:16.878902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09486e-770a-4467-a18e-507c4e4d7754 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:16.879180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f09486e-770a-4467-a18e-507c4e4d7754 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:16.879905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09486e-770a-4467-a18e-507c4e4d7754 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:16.880215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09486e-770a-4467-a18e-507c4e4d7754 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:16.881569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f09486e-770a-4467-a18e-507c4e4d7754 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:16.898775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 508/1030] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:16.900372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8a662-7a19-4489-bfc9-a6d16e5d0965 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:16.953035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:16.953801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1816ab-e2ed-4839-b4ee-c910d97b72f0 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:16.954134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1816ab-e2ed-4839-b4ee-c910d97b72f0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:16.954595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1816ab-e2ed-4839-b4ee-c910d97b72f0 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:16.956672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 509/1031] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:16.961920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a5ebb-7ee3-414d-8fc7-bc53e676a520 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:16.962236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a5ebb-7ee3-414d-8fc7-bc53e676a520 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:16.962519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a5ebb-7ee3-414d-8fc7-bc53e676a520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:16.979226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-299a5ebb-7ee3-414d-8fc7-bc53e676a520 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:16.980789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-299a5ebb-7ee3-414d-8fc7-bc53e676a520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:17.016416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c8a662-7a19-4489-bfc9-a6d16e5d0965 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:17.055015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce3354-c1c5-4aaa-9a32-8b6621510cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:17.055807 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce3354-c1c5-4aaa-9a32-8b6621510cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:17.056092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce3354-c1c5-4aaa-9a32-8b6621510cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:17.056913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce3354-c1c5-4aaa-9a32-8b6621510cf8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:17.062208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 510/1032] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:17.064493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce3354-c1c5-4aaa-9a32-8b6621510cf8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:17.065907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce3354-c1c5-4aaa-9a32-8b6621510cf8 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:17.074049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a5ebb-7ee3-414d-8fc7-bc53e676a520 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-53ce3354-c1c5-4aaa-9a32-8b6621510cf8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:53:17.079853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:17.080754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:17.081679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 523/1033] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Jun 26 02:53:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:17.086615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce3354-c1c5-4aaa-9a32-8b6621510cf8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:17.182797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a5ebb-7ee3-414d-8fc7-bc53e676a520 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:17.193313 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce3354-c1c5-4aaa-9a32-8b6621510cf8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:17.199939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:17.204742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:17.219223 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba6030-4b42-454a-900a-b2d332acf587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:17.220009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba6030-4b42-454a-900a-b2d332acf587 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:17.220293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba6030-4b42-454a-900a-b2d332acf587 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:17.220568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba6030-4b42-454a-900a-b2d332acf587 None admin] PATH_INFO: `/v3/services/image` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:17.221216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba6030-4b42-454a-900a-b2d332acf587 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:17.222602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba6030-4b42-454a-900a-b2d332acf587 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:17.230647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59036585-9b0f-481a-a568-fc0ef3ba4448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:17.231266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59036585-9b0f-481a-a568-fc0ef3ba4448 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:17.231558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59036585-9b0f-481a-a568-fc0ef3ba4448 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:17.231840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59036585-9b0f-481a-a568-fc0ef3ba4448 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:17.232762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59036585-9b0f-481a-a568-fc0ef3ba4448 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:17.233085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59036585-9b0f-481a-a568-fc0ef3ba4448 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:17.234547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59036585-9b0f-481a-a568-fc0ef3ba4448 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-41ba6030-4b42-454a-900a-b2d332acf587 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:17.234850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:17.235763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:17.236583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 511/1034] 10.222.0.13 () {56 vars in 1105 bytes} [Sun Jun 26 02:53:17 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:17.241990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba6030-4b42-454a-900a-b2d332acf587 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:17.257891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 524/1035] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Jun 26 02:53:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:17.263480 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59036585-9b0f-481a-a568-fc0ef3ba4448 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:17.351768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba6030-4b42-454a-900a-b2d332acf587 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:17.371763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59036585-9b0f-481a-a568-fc0ef3ba4448 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:17.386479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813611-bc29-42b2-ab98-ca13d3c6b70f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:17.387138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813611-bc29-42b2-ab98-ca13d3c6b70f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:17.387416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813611-bc29-42b2-ab98-ca13d3c6b70f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:17.387690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71813611-bc29-42b2-ab98-ca13d3c6b70f None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:17.388379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71813611-bc29-42b2-ab98-ca13d3c6b70f None admin] RBAC: Adding query filter params (name=image) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:17.388682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71813611-bc29-42b2-ab98-ca13d3c6b70f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:17.390028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71813611-bc29-42b2-ab98-ca13d3c6b70f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:17.399465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 512/1036] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:17.400935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813611-bc29-42b2-ab98-ca13d3c6b70f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:17.412425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37476133-82bf-4f8c-807b-553d20a5fcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:17.413080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37476133-82bf-4f8c-807b-553d20a5fcca None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:17.413366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37476133-82bf-4f8c-807b-553d20a5fcca None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:17.413641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37476133-82bf-4f8c-807b-553d20a5fcca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:17.449034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:17.451877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:17.487611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37476133-82bf-4f8c-807b-553d20a5fcca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:17.488984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37476133-82bf-4f8c-807b-553d20a5fcca None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:17.513056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71813611-bc29-42b2-ab98-ca13d3c6b70f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:17.551659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e46710-7ea3-4d03-9f1f-b43ce1679aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:17.552321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e46710-7ea3-4d03-9f1f-b43ce1679aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:17.552608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e46710-7ea3-4d03-9f1f-b43ce1679aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:17.552886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e46710-7ea3-4d03-9f1f-b43ce1679aa4 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:17.553591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e46710-7ea3-4d03-9f1f-b43ce1679aa4 None admin] RBAC: Adding query filter params (type=image) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:17.553899 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e46710-7ea3-4d03-9f1f-b43ce1679aa4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:17.555291 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e46710-7ea3-4d03-9f1f-b43ce1679aa4 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:17.560643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 513/1037] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:17.566173 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e46710-7ea3-4d03-9f1f-b43ce1679aa4 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:17.601576 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 525/1038] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:17 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 26 02:53:17.608173 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37476133-82bf-4f8c-807b-553d20a5fcca None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:17.674732 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e46710-7ea3-4d03-9f1f-b43ce1679aa4 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:17.679229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:17.684508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:17.713188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d8e09-2333-4454-93fe-4624e04be565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:17.713844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d8e09-2333-4454-93fe-4624e04be565 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:17.714157 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d8e09-2333-4454-93fe-4624e04be565 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:17.714435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d8e09-2333-4454-93fe-4624e04be565 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:17.714787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37476133-82bf-4f8c-807b-553d20a5fcca None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:17.715216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d8e09-2333-4454-93fe-4624e04be565 None admin] RBAC: Adding query filter params (interface=public, service_id=53e5496368334d25b5727d06f85b4c54, region_id=RegionOne) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:17.715526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d8e09-2333-4454-93fe-4624e04be565 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:17.716870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d8e09-2333-4454-93fe-4624e04be565 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:17.722092 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 514/1039] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Jun 26 02:53:17 2022] GET /identity/v3/endpoints?service_id=53e5496368334d25b5727d06f85b4c54&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:17.727696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d8e09-2333-4454-93fe-4624e04be565 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:17.752052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050adeb5-4c31-4d5b-bd49-248bd479abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:17.752773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050adeb5-4c31-4d5b-bd49-248bd479abcf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:17.753057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050adeb5-4c31-4d5b-bd49-248bd479abcf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:17.753333 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050adeb5-4c31-4d5b-bd49-248bd479abcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:17.764596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050adeb5-4c31-4d5b-bd49-248bd479abcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:17.766035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050adeb5-4c31-4d5b-bd49-248bd479abcf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-050adeb5-4c31-4d5b-bd49-248bd479abcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:17.785117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:17.786023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:17.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:17.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:17.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:17.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:17.786939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 526/1040] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:17.792328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050adeb5-4c31-4d5b-bd49-248bd479abcf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:17.840475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d8e09-2333-4454-93fe-4624e04be565 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:17.879970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf3bb5-316e-4ad1-bb7c-2f6fe9fe1c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:17.880634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf3bb5-316e-4ad1-bb7c-2f6fe9fe1c57 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:17.880915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf3bb5-316e-4ad1-bb7c-2f6fe9fe1c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:17.881210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edf3bb5-316e-4ad1-bb7c-2f6fe9fe1c57 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:17.881858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf3bb5-316e-4ad1-bb7c-2f6fe9fe1c57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:17.883276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edf3bb5-316e-4ad1-bb7c-2f6fe9fe1c57 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:17.902072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 515/1041] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Jun 26 02:53:17 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 02:53:17.914798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050adeb5-4c31-4d5b-bd49-248bd479abcf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:17.938415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:17.955645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df48e5-90d5-4abd-aad0-581a1fa981f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:17.956398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df48e5-90d5-4abd-aad0-581a1fa981f1 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:17.956685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df48e5-90d5-4abd-aad0-581a1fa981f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:17.956959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08df48e5-90d5-4abd-aad0-581a1fa981f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:17.957757 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df48e5-90d5-4abd-aad0-581a1fa981f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:17.958129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df48e5-90d5-4abd-aad0-581a1fa981f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:17.959497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08df48e5-90d5-4abd-aad0-581a1fa981f1 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:17.977583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 527/1042] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:17.978933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf3bb5-316e-4ad1-bb7c-2f6fe9fe1c57 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:18.018402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:18.091788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edf3bb5-316e-4ad1-bb7c-2f6fe9fe1c57 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:18.125999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce748c-8322-49e7-90c0-bfdfc1bb88eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:18.126815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce748c-8322-49e7-90c0-bfdfc1bb88eb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:18.127099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce748c-8322-49e7-90c0-bfdfc1bb88eb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:18.127375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce748c-8322-49e7-90c0-bfdfc1bb88eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:18.134188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce748c-8322-49e7-90c0-bfdfc1bb88eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:18.136104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce748c-8322-49e7-90c0-bfdfc1bb88eb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-01ce748c-8322-49e7-90c0-bfdfc1bb88eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:53:18.150902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:18.151802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:18.152690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:53:18.152690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:53:18.152690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:53:18.152690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:53:18.152690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:53:18.152690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:53:18.152690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:18.152690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:18.152690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 516/1043] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:53:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:18.159309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df48e5-90d5-4abd-aad0-581a1fa981f1 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:18.180327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b32284-97db-460f-ae5b-38d9ed2b01d6 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:18.180616 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b32284-97db-460f-ae5b-38d9ed2b01d6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:18.180896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b32284-97db-460f-ae5b-38d9ed2b01d6 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:18.182988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 517/1044] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:18.187582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769ef8c-b8e1-4079-86a2-b5190c0d8f63 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:18.187868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769ef8c-b8e1-4079-86a2-b5190c0d8f63 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:18.188253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769ef8c-b8e1-4079-86a2-b5190c0d8f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:18.205079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3769ef8c-b8e1-4079-86a2-b5190c0d8f63 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:18.206142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3769ef8c-b8e1-4079-86a2-b5190c0d8f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:18.268526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08df48e5-90d5-4abd-aad0-581a1fa981f1 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:18.278106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 518/1045] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:18.299940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769ef8c-b8e1-4079-86a2-b5190c0d8f63 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:18.303163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a7e1b-d3d1-42ea-872c-92166b4c02d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:18.303832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a7e1b-d3d1-42ea-872c-92166b4c02d2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:18.304117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a7e1b-d3d1-42ea-872c-92166b4c02d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:18.304395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2a7e1b-d3d1-42ea-872c-92166b4c02d2 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:18.305182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a7e1b-d3d1-42ea-872c-92166b4c02d2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:18.305492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a7e1b-d3d1-42ea-872c-92166b4c02d2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:18.307430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2a7e1b-d3d1-42ea-872c-92166b4c02d2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:18.325651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 528/1046] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:53:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:18.326714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a7e1b-d3d1-42ea-872c-92166b4c02d2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:18.333058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:18.373901 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:18.409375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769ef8c-b8e1-4079-86a2-b5190c0d8f63 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:18.435854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2a7e1b-d3d1-42ea-872c-92166b4c02d2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:18.444231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75133bbb-3659-4932-8809-9db31e503506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:18.445016 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75133bbb-3659-4932-8809-9db31e503506 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:18.445297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75133bbb-3659-4932-8809-9db31e503506 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:18.445665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75133bbb-3659-4932-8809-9db31e503506 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:18.456439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75133bbb-3659-4932-8809-9db31e503506 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-75133bbb-3659-4932-8809-9db31e503506 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:18.464093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:18.465046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:18.466098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 519/1047] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:18.471053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75133bbb-3659-4932-8809-9db31e503506 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:18.474331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b39cb3-7c62-40ee-a3dc-0ae9d8697748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:18.474992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b39cb3-7c62-40ee-a3dc-0ae9d8697748 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:18.475272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b39cb3-7c62-40ee-a3dc-0ae9d8697748 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:18.475546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b39cb3-7c62-40ee-a3dc-0ae9d8697748 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:18.476370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b39cb3-7c62-40ee-a3dc-0ae9d8697748 None admin] RBAC: Adding query filter params (role.id=d34bc405b0c04033a57dc7dae296fdee, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=9ee0150dfb3f42709029e42d0fd9105d) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:18.476684 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b39cb3-7c62-40ee-a3dc-0ae9d8697748 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:18.478163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b39cb3-7c62-40ee-a3dc-0ae9d8697748 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:18.484086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 529/1048] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:53:18 2022] GET /identity/v3/role_assignments?user.id=9ee0150dfb3f42709029e42d0fd9105d&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=d34bc405b0c04033a57dc7dae296fdee => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:18.564403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:18.581855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75133bbb-3659-4932-8809-9db31e503506 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:18.618405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27090748-466f-490c-9fab-e2ce12d19790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:18.619701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090748-466f-490c-9fab-e2ce12d19790 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:18.619958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090748-466f-490c-9fab-e2ce12d19790 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:18.620212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090748-466f-490c-9fab-e2ce12d19790 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:18.620976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27090748-466f-490c-9fab-e2ce12d19790 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:18.621286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27090748-466f-490c-9fab-e2ce12d19790 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:18.622704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27090748-466f-490c-9fab-e2ce12d19790 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:18.632343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 520/1049] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:18.637857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b39cb3-7c62-40ee-a3dc-0ae9d8697748 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:18.737687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b39cb3-7c62-40ee-a3dc-0ae9d8697748 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:18.740043 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:18.773116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae277b-df0a-4279-b235-a380e0606d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:18.773980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae277b-df0a-4279-b235-a380e0606d62 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:18.774287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae277b-df0a-4279-b235-a380e0606d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:18.774559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ae277b-df0a-4279-b235-a380e0606d62 None admin] PATH_INFO: `/v3/users/50134745807f42f1ab662f7d2fea69c7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:18.787875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae277b-df0a-4279-b235-a380e0606d62 None admin] RBAC: Authorizing `identity:get_user(user_id=50134745807f42f1ab662f7d2fea69c7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:18.789315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ae277b-df0a-4279-b235-a380e0606d62 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:18.791784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 530/1050] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:18 2022] GET /identity/v3/users/50134745807f42f1ab662f7d2fea69c7 => generated 285 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:18.797631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27090748-466f-490c-9fab-e2ce12d19790 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:18.904694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27090748-466f-490c-9fab-e2ce12d19790 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:18.940738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97910f6-cdd1-426a-bc80-c9ba43a60354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:18.941393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97910f6-cdd1-426a-bc80-c9ba43a60354 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:18.941679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97910f6-cdd1-426a-bc80-c9ba43a60354 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:18.941955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97910f6-cdd1-426a-bc80-c9ba43a60354 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:18.960257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:19.015911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97910f6-cdd1-426a-bc80-c9ba43a60354 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:19.017397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97910f6-cdd1-426a-bc80-c9ba43a60354 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:19.117125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 521/1051] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:18 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 26 02:53:19.122610 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae277b-df0a-4279-b235-a380e0606d62 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:19.221484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ae277b-df0a-4279-b235-a380e0606d62 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:19.254979 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52903546-6cf5-4f8d-8afc-ec56c8f885c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:19.255838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52903546-6cf5-4f8d-8afc-ec56c8f885c2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:19.256121 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52903546-6cf5-4f8d-8afc-ec56c8f885c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:19.256397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52903546-6cf5-4f8d-8afc-ec56c8f885c2 None admin] PATH_INFO: `/v3/projects/628baa9ce0074b99ba0dd86a7223ae81` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:19.257565 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52903546-6cf5-4f8d-8afc-ec56c8f885c2 None admin] RBAC: Authorizing `identity:get_project(project_id=628baa9ce0074b99ba0dd86a7223ae81)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:19.259016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52903546-6cf5-4f8d-8afc-ec56c8f885c2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:19.260928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 531/1052] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:53:19 2022] GET /identity/v3/projects/628baa9ce0074b99ba0dd86a7223ae81 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:19.266736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97910f6-cdd1-426a-bc80-c9ba43a60354 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:19.314851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:19.367520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97910f6-cdd1-426a-bc80-c9ba43a60354 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:19.401729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229ce61-80ff-4b93-8c49-819e985e5d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:19.402406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229ce61-80ff-4b93-8c49-819e985e5d8c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:19.402694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229ce61-80ff-4b93-8c49-819e985e5d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:19.402968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229ce61-80ff-4b93-8c49-819e985e5d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:19.403797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229ce61-80ff-4b93-8c49-819e985e5d8c None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=628baa9ce0074b99ba0dd86a7223ae81, user.id=50134745807f42f1ab662f7d2fea69c7) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:19.404112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229ce61-80ff-4b93-8c49-819e985e5d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:19.405531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c229ce61-80ff-4b93-8c49-819e985e5d8c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:19.417964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 522/1053] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:53:19 2022] GET /identity/v3/role_assignments?user.id=50134745807f42f1ab662f7d2fea69c7&scope.project.id=628baa9ce0074b99ba0dd86a7223ae81&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:19.960248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5c9f1-27a6-4dec-a300-f50558b8a277 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:19.960776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5c9f1-27a6-4dec-a300-f50558b8a277 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:19.960776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5c9f1-27a6-4dec-a300-f50558b8a277 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:19.962860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 532/1054] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:19.967906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc704c03-6647-4f5f-b506-9a41218d6b8b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:19.968276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc704c03-6647-4f5f-b506-9a41218d6b8b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:19.968480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc704c03-6647-4f5f-b506-9a41218d6b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:19.985371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cc704c03-6647-4f5f-b506-9a41218d6b8b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:19.986588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cc704c03-6647-4f5f-b506-9a41218d6b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:20.055647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 523/1055] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:20.083002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5c9f1-27a6-4dec-a300-f50558b8a277 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:20.114133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:20.187288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5c9f1-27a6-4dec-a300-f50558b8a277 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:20.221528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0b2a5-4c8c-4143-b745-87477e3c42ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:20.222352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b2a5-4c8c-4143-b745-87477e3c42ed None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:20.222637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b2a5-4c8c-4143-b745-87477e3c42ed None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:20.222920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0b2a5-4c8c-4143-b745-87477e3c42ed None admin] PATH_INFO: `/v3/services/image` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:20.223575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0b2a5-4c8c-4143-b745-87477e3c42ed None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:20.224940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c0b2a5-4c8c-4143-b745-87477e3c42ed None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-99c0b2a5-4c8c-4143-b745-87477e3c42ed None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:20.236431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:20.237409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 26 02:53:20.238656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:20.242395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 533/1056] 10.222.0.13 () {56 vars in 1105 bytes} [Sun Jun 26 02:53:20 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:20.243618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc704c03-6647-4f5f-b506-9a41218d6b8b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:20.287828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:20.346274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc704c03-6647-4f5f-b506-9a41218d6b8b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:20.382210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af259dd7-7ec5-4fec-8fa0-b020f1e3eab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:20.382887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af259dd7-7ec5-4fec-8fa0-b020f1e3eab9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:20.383172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af259dd7-7ec5-4fec-8fa0-b020f1e3eab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:20.383447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af259dd7-7ec5-4fec-8fa0-b020f1e3eab9 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:20.384155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af259dd7-7ec5-4fec-8fa0-b020f1e3eab9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:20.385034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af259dd7-7ec5-4fec-8fa0-b020f1e3eab9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:20.386399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af259dd7-7ec5-4fec-8fa0-b020f1e3eab9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:20.396008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 524/1057] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:20.397554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0b2a5-4c8c-4143-b745-87477e3c42ed None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:20.496025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:20.504426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0b2a5-4c8c-4143-b745-87477e3c42ed None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:20.517903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c485beb-3dc9-419e-a4a0-d322e06c5e07 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:20.518232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c485beb-3dc9-419e-a4a0-d322e06c5e07 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:20.518512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c485beb-3dc9-419e-a4a0-d322e06c5e07 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:20.520033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 525/1058] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:20.524581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac2e37-830f-48b5-8382-615623447ea3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:20.524899 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac2e37-830f-48b5-8382-615623447ea3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:20.525177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac2e37-830f-48b5-8382-615623447ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:20.541573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb685e6b-25fb-4c1b-88de-f34cd9318dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:20.542279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb685e6b-25fb-4c1b-88de-f34cd9318dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:20.542562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb685e6b-25fb-4c1b-88de-f34cd9318dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:20.542811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b2ac2e37-830f-48b5-8382-615623447ea3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:20.542914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb685e6b-25fb-4c1b-88de-f34cd9318dc0 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:20.543521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb685e6b-25fb-4c1b-88de-f34cd9318dc0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:20.543826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb685e6b-25fb-4c1b-88de-f34cd9318dc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:20.543988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b2ac2e37-830f-48b5-8382-615623447ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:20.545170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb685e6b-25fb-4c1b-88de-f34cd9318dc0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:20.550645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 534/1059] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:20.556968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb685e6b-25fb-4c1b-88de-f34cd9318dc0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:20.614653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 526/1060] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:20.638781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac2e37-830f-48b5-8382-615623447ea3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:20.662687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb685e6b-25fb-4c1b-88de-f34cd9318dc0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:20.673712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:20.700045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc143168-a075-4893-a507-86a7ad9f0779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:20.700712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc143168-a075-4893-a507-86a7ad9f0779 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:20.700993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc143168-a075-4893-a507-86a7ad9f0779 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:20.701270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc143168-a075-4893-a507-86a7ad9f0779 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:20.702003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc143168-a075-4893-a507-86a7ad9f0779 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:20.703382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc143168-a075-4893-a507-86a7ad9f0779 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:20.730290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 535/1061] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Jun 26 02:53:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:53:20.753112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac2e37-830f-48b5-8382-615623447ea3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:20.797662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135ca11-f1bf-4462-8737-f7f0c8aeea87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:20.798516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135ca11-f1bf-4462-8737-f7f0c8aeea87 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:20.798800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135ca11-f1bf-4462-8737-f7f0c8aeea87 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:20.799072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0135ca11-f1bf-4462-8737-f7f0c8aeea87 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:20.811285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0135ca11-f1bf-4462-8737-f7f0c8aeea87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-0135ca11-f1bf-4462-8737-f7f0c8aeea87 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:20.819748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:20.820770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:53:20.821581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:20.826498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 527/1062] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Jun 26 02:53:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:20.827352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc143168-a075-4893-a507-86a7ad9f0779 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:20.855134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:20.946501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc143168-a075-4893-a507-86a7ad9f0779 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:20.980655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948262c9-b7ce-4f66-a060-6e3e43015439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:20.981396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948262c9-b7ce-4f66-a060-6e3e43015439 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:20.981605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948262c9-b7ce-4f66-a060-6e3e43015439 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:20.981883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948262c9-b7ce-4f66-a060-6e3e43015439 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:20.982659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948262c9-b7ce-4f66-a060-6e3e43015439 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:20.982969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948262c9-b7ce-4f66-a060-6e3e43015439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:20.984332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948262c9-b7ce-4f66-a060-6e3e43015439 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:20.992859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 536/1063] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Jun 26 02:53:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:20.998921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135ca11-f1bf-4462-8737-f7f0c8aeea87 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:21.091147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:21.101021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0135ca11-f1bf-4462-8737-f7f0c8aeea87 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:21.135301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3ce9c-6639-4979-ba5e-bc2a69254ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:21.136087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3ce9c-6639-4979-ba5e-bc2a69254ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:21.136366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3ce9c-6639-4979-ba5e-bc2a69254ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:21.136643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b3ce9c-6639-4979-ba5e-bc2a69254ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:21.146994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3ce9c-6639-4979-ba5e-bc2a69254ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:21.148422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b3ce9c-6639-4979-ba5e-bc2a69254ce7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-87b3ce9c-6639-4979-ba5e-bc2a69254ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:21.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:21.167089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:21.167968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:21.167968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:21.167968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:21.167968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:21.167968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 528/1064] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:21.173948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948262c9-b7ce-4f66-a060-6e3e43015439 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:21.255434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:21.280075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948262c9-b7ce-4f66-a060-6e3e43015439 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:21.313683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff084c-12e8-443d-b244-8b9d70e1022d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:21.314374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff084c-12e8-443d-b244-8b9d70e1022d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:21.314662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff084c-12e8-443d-b244-8b9d70e1022d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:21.314938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff084c-12e8-443d-b244-8b9d70e1022d None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:21.315654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff084c-12e8-443d-b244-8b9d70e1022d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:21.315964 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff084c-12e8-443d-b244-8b9d70e1022d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:21.317322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff084c-12e8-443d-b244-8b9d70e1022d None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:21.335360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 537/1065] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:21.336632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3ce9c-6639-4979-ba5e-bc2a69254ce7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:21.425755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4272f8-de41-4aec-b2bb-ca0cd88d4429 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:21.426392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4272f8-de41-4aec-b2bb-ca0cd88d4429 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:21.426860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4272f8-de41-4aec-b2bb-ca0cd88d4429 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:21.429191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 538/1066] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:21.435174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41f6ae-0d3c-4538-aa59-5a34e39ed326 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:21.435768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41f6ae-0d3c-4538-aa59-5a34e39ed326 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:21.436235 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41f6ae-0d3c-4538-aa59-5a34e39ed326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:21.440236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b3ce9c-6639-4979-ba5e-bc2a69254ce7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:21.458395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6a41f6ae-0d3c-4538-aa59-5a34e39ed326 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:21.459944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6a41f6ae-0d3c-4538-aa59-5a34e39ed326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:21.475404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bbcbf-c58f-4a74-99ff-44636e737ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:21.476199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bbcbf-c58f-4a74-99ff-44636e737ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:21.476489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bbcbf-c58f-4a74-99ff-44636e737ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:21.476761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5bbcbf-c58f-4a74-99ff-44636e737ae5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:21.485127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bbcbf-c58f-4a74-99ff-44636e737ae5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:21.486602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5bbcbf-c58f-4a74-99ff-44636e737ae5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ab5bbcbf-c58f-4a74-99ff-44636e737ae5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:53:21.500874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:21.501813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:21.503231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 529/1067] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Jun 26 02:53:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:21.507660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bbcbf-c58f-4a74-99ff-44636e737ae5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:21.511944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:21.543449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 539/1068] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:21.565653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41f6ae-0d3c-4538-aa59-5a34e39ed326 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:21.620318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5bbcbf-c58f-4a74-99ff-44636e737ae5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:21.632894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:21.657521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a251e4f-b986-43b7-9826-fa319ddc5653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:21.657985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a251e4f-b986-43b7-9826-fa319ddc5653 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:21.658303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a251e4f-b986-43b7-9826-fa319ddc5653 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:21.658581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a251e4f-b986-43b7-9826-fa319ddc5653 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:21.659444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a251e4f-b986-43b7-9826-fa319ddc5653 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:21.659760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a251e4f-b986-43b7-9826-fa319ddc5653 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:21.661175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a251e4f-b986-43b7-9826-fa319ddc5653 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:21.670938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41f6ae-0d3c-4538-aa59-5a34e39ed326 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:21.678406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 530/1069] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Jun 26 02:53:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:21.683923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a251e4f-b986-43b7-9826-fa319ddc5653 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:21.709717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169909b-ad50-4582-afe9-e5b10583e170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:21.710550 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169909b-ad50-4582-afe9-e5b10583e170 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:21.710836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169909b-ad50-4582-afe9-e5b10583e170 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:21.711118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169909b-ad50-4582-afe9-e5b10583e170 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:21.722002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169909b-ad50-4582-afe9-e5b10583e170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5169909b-ad50-4582-afe9-e5b10583e170 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:21.730798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:21.731756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:21.732589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:21.737179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 540/1070] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:53:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:21.737803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169909b-ad50-4582-afe9-e5b10583e170 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:21.749537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:21.793461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a251e4f-b986-43b7-9826-fa319ddc5653 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:21.829140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634a30c-7647-4d81-93c0-8dc91be741fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:21.829933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634a30c-7647-4d81-93c0-8dc91be741fc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:21.830246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634a30c-7647-4d81-93c0-8dc91be741fc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:21.830520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634a30c-7647-4d81-93c0-8dc91be741fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:21.840754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169909b-ad50-4582-afe9-e5b10583e170 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:21.841054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634a30c-7647-4d81-93c0-8dc91be741fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:21.842530 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634a30c-7647-4d81-93c0-8dc91be741fc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-e634a30c-7647-4d81-93c0-8dc91be741fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:21.862577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:21.863576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:21.864443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:21.864443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:21.864443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:21.864443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:21.868826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 531/1071] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:21.869320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634a30c-7647-4d81-93c0-8dc91be741fc None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:21.882492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:21.895531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ded358-619f-4ebf-b27c-9573bb583429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:21.896456 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ded358-619f-4ebf-b27c-9573bb583429 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:21.896747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ded358-619f-4ebf-b27c-9573bb583429 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:21.897263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ded358-619f-4ebf-b27c-9573bb583429 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:21.898379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ded358-619f-4ebf-b27c-9573bb583429 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:21.898706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ded358-619f-4ebf-b27c-9573bb583429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:21.900682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ded358-619f-4ebf-b27c-9573bb583429 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:21.914378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 541/1072] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:53:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:21.916584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ded358-619f-4ebf-b27c-9573bb583429 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:21.977494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634a30c-7647-4d81-93c0-8dc91be741fc None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:21.991905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:22.016148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d13b0-c6fc-405f-ba6c-7e2f45bffe04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:22.016922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d13b0-c6fc-405f-ba6c-7e2f45bffe04 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:22.017233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d13b0-c6fc-405f-ba6c-7e2f45bffe04 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:22.017482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4d13b0-c6fc-405f-ba6c-7e2f45bffe04 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:22.018228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d13b0-c6fc-405f-ba6c-7e2f45bffe04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:22.018542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d13b0-c6fc-405f-ba6c-7e2f45bffe04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:22.019899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ded358-619f-4ebf-b27c-9573bb583429 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:22.020346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4d13b0-c6fc-405f-ba6c-7e2f45bffe04 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:22.037684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 532/1073] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:22.038591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d13b0-c6fc-405f-ba6c-7e2f45bffe04 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:22.053266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7f443-b649-4ef6-a0e9-c0f641a948ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:22.054091 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7f443-b649-4ef6-a0e9-c0f641a948ba None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:22.054396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7f443-b649-4ef6-a0e9-c0f641a948ba None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:22.054666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7f443-b649-4ef6-a0e9-c0f641a948ba None admin] PATH_INFO: `/v3/users/50134745807f42f1ab662f7d2fea69c7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:22.055875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7f443-b649-4ef6-a0e9-c0f641a948ba None admin] RBAC: Authorizing `identity:get_user(user_id=50134745807f42f1ab662f7d2fea69c7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:22.057833 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7f443-b649-4ef6-a0e9-c0f641a948ba None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:22.059514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 542/1074] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:21 2022] GET /identity/v3/users/50134745807f42f1ab662f7d2fea69c7 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:22.064984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7f443-b649-4ef6-a0e9-c0f641a948ba None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:22.126322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:22.142261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4d13b0-c6fc-405f-ba6c-7e2f45bffe04 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:22.172189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7f443-b649-4ef6-a0e9-c0f641a948ba None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:22.175404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f470b-802f-4d36-8ff9-3f698b44cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:22.176201 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f470b-802f-4d36-8ff9-3f698b44cb48 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:22.176483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f470b-802f-4d36-8ff9-3f698b44cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:22.176758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f470b-802f-4d36-8ff9-3f698b44cb48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:22.183973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f470b-802f-4d36-8ff9-3f698b44cb48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:22.185428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f470b-802f-4d36-8ff9-3f698b44cb48 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-8b2f470b-802f-4d36-8ff9-3f698b44cb48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:53:22.200015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:22.200944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:22.201912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:53:22.201912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:53:22.201912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:53:22.201912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:53:22.201912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:53:22.201912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:53:22.201912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:22.201912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:22.201912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 533/1075] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:53:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:22.206946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f470b-802f-4d36-8ff9-3f698b44cb48 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:22.214132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b59461-ce6c-49fb-b5a8-1a1efab99f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:22.215171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b59461-ce6c-49fb-b5a8-1a1efab99f91 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:22.215619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b59461-ce6c-49fb-b5a8-1a1efab99f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:22.216058 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b59461-ce6c-49fb-b5a8-1a1efab99f91 None admin] PATH_INFO: `/v3/projects/628baa9ce0074b99ba0dd86a7223ae81` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:22.217409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b59461-ce6c-49fb-b5a8-1a1efab99f91 None admin] RBAC: Authorizing `identity:get_project(project_id=628baa9ce0074b99ba0dd86a7223ae81)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:22.218890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b59461-ce6c-49fb-b5a8-1a1efab99f91 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:22.220914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 543/1076] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:53:22 2022] GET /identity/v3/projects/628baa9ce0074b99ba0dd86a7223ae81 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:22.226357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b59461-ce6c-49fb-b5a8-1a1efab99f91 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:22.245741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:22.320558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f470b-802f-4d36-8ff9-3f698b44cb48 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:22.334818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b59461-ce6c-49fb-b5a8-1a1efab99f91 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:22.357574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5430f2-0318-48a1-bf88-de93308589e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:22.358602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5430f2-0318-48a1-bf88-de93308589e9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:22.359105 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5430f2-0318-48a1-bf88-de93308589e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:22.359449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5430f2-0318-48a1-bf88-de93308589e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:22.360240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5430f2-0318-48a1-bf88-de93308589e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:22.360549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5430f2-0318-48a1-bf88-de93308589e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:22.360651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:22.362094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5430f2-0318-48a1-bf88-de93308589e9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:22.374785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 534/1077] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:53:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:22.377253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7b1abc-041d-4761-84ed-977460c4c654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:22.378176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b1abc-041d-4761-84ed-977460c4c654 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:22.378458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b1abc-041d-4761-84ed-977460c4c654 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:22.378729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7b1abc-041d-4761-84ed-977460c4c654 None admin] PATH_INFO: `/v3/projects/628baa9ce0074b99ba0dd86a7223ae81/users/50134745807f42f1ab662f7d2fea69c7/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:22.380636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5430f2-0318-48a1-bf88-de93308589e9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:22.380777 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b1abc-041d-4761-84ed-977460c4c654 None admin] RBAC: Authorizing `identity:create_grant(project_id=628baa9ce0074b99ba0dd86a7223ae81, user_id=50134745807f42f1ab662f7d2fea69c7, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:22.382222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7b1abc-041d-4761-84ed-977460c4c654 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:22.397120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 544/1078] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:53:22 2022] PUT /identity/v3/projects/628baa9ce0074b99ba0dd86a7223ae81/users/50134745807f42f1ab662f7d2fea69c7/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 02:53:22.486643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:22.495572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5430f2-0318-48a1-bf88-de93308589e9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:22.531021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f7c91-be9a-46f5-8ac9-56e2f178682a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:22.531929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f7c91-be9a-46f5-8ac9-56e2f178682a None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:22.532268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f7c91-be9a-46f5-8ac9-56e2f178682a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:22.532551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f7c91-be9a-46f5-8ac9-56e2f178682a None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/9ee0150dfb3f42709029e42d0fd9105d/roles/d34bc405b0c04033a57dc7dae296fdee` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:22.534888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f7c91-be9a-46f5-8ac9-56e2f178682a None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=9ee0150dfb3f42709029e42d0fd9105d, role_id=d34bc405b0c04033a57dc7dae296fdee)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:22.536335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f7c91-be9a-46f5-8ac9-56e2f178682a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:22.554759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 535/1079] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:53:22 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/9ee0150dfb3f42709029e42d0fd9105d/roles/d34bc405b0c04033a57dc7dae296fdee => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:53:22.784962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b556f-5e06-45fb-ae2f-113ba4034690 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:22.785365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b556f-5e06-45fb-ae2f-113ba4034690 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:22.785545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5b556f-5e06-45fb-ae2f-113ba4034690 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:22.787114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 545/1080] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:22.792590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88216f12-c44b-4992-9dc6-569e5ba75ee1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:22.792888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88216f12-c44b-4992-9dc6-569e5ba75ee1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:22.793170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88216f12-c44b-4992-9dc6-569e5ba75ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:22.809863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-88216f12-c44b-4992-9dc6-569e5ba75ee1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:22.811073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-88216f12-c44b-4992-9dc6-569e5ba75ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:22.885475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 536/1081] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:22.908297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b556f-5e06-45fb-ae2f-113ba4034690 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:23.011549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5b556f-5e06-45fb-ae2f-113ba4034690 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:23.045700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7cdd-0ef0-42e6-916e-dea9d4348d92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:23.046325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7cdd-0ef0-42e6-916e-dea9d4348d92 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:23.046567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7cdd-0ef0-42e6-916e-dea9d4348d92 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:23.046839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dc7cdd-0ef0-42e6-916e-dea9d4348d92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:23.098254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:23.116369 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc7cdd-0ef0-42e6-916e-dea9d4348d92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:23.117803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dc7cdd-0ef0-42e6-916e-dea9d4348d92 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:23.219237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 546/1082] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:53:23.221069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88216f12-c44b-4992-9dc6-569e5ba75ee1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:23.305429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:23.323575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88216f12-c44b-4992-9dc6-569e5ba75ee1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:23.356670 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5b445-934c-49d8-91ab-cfefd48c4a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:23.357482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5b445-934c-49d8-91ab-cfefd48c4a24 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:23.357824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5b445-934c-49d8-91ab-cfefd48c4a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:23.358143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5b445-934c-49d8-91ab-cfefd48c4a24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:23.367989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5b445-934c-49d8-91ab-cfefd48c4a24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:23.370083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5b445-934c-49d8-91ab-cfefd48c4a24 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-31d5b445-934c-49d8-91ab-cfefd48c4a24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:23.387192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:23.388049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:23.388911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:23.388911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:23.388911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:23.388911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:23.390355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 537/1083] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:23.394320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7cdd-0ef0-42e6-916e-dea9d4348d92 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:23.476840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:23.494464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dc7cdd-0ef0-42e6-916e-dea9d4348d92 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:23.530669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8886695-f038-48cb-9487-3f74dfdd8058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:23.531296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8886695-f038-48cb-9487-3f74dfdd8058 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:23.531583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8886695-f038-48cb-9487-3f74dfdd8058 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:23.531858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8886695-f038-48cb-9487-3f74dfdd8058 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:23.532579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8886695-f038-48cb-9487-3f74dfdd8058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:23.532886 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8886695-f038-48cb-9487-3f74dfdd8058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:23.534268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8886695-f038-48cb-9487-3f74dfdd8058 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:23.546264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 547/1084] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:24.412788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9160fa-64fa-42fc-bd7e-d14e9078685d None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:24.413311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9160fa-64fa-42fc-bd7e-d14e9078685d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:24.413425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9160fa-64fa-42fc-bd7e-d14e9078685d None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:24.414870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 538/1085] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:24.420128 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf373736-0c2f-462b-9019-6fea78c031d3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:24.420517 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf373736-0c2f-462b-9019-6fea78c031d3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:24.420809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf373736-0c2f-462b-9019-6fea78c031d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:24.437768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cf373736-0c2f-462b-9019-6fea78c031d3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:24.438987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cf373736-0c2f-462b-9019-6fea78c031d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:24.503161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 548/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:24.526985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9160fa-64fa-42fc-bd7e-d14e9078685d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:24.590071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e261748-9e0f-43c6-983d-fd59f6fbdf73 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:24.590364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e261748-9e0f-43c6-983d-fd59f6fbdf73 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:24.590645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e261748-9e0f-43c6-983d-fd59f6fbdf73 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:24.592180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 549/1087] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:53:24.597181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388edff-8a2c-42ca-aee0-97150f56f802 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:24.597463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388edff-8a2c-42ca-aee0-97150f56f802 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:24.597736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2388edff-8a2c-42ca-aee0-97150f56f802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:24.614409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2388edff-8a2c-42ca-aee0-97150f56f802 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:24.615606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2388edff-8a2c-42ca-aee0-97150f56f802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:24.632629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9160fa-64fa-42fc-bd7e-d14e9078685d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:24.665264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f04b73-5150-49aa-8ecd-e0cb16742036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:24.666542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f04b73-5150-49aa-8ecd-e0cb16742036 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:24.666826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f04b73-5150-49aa-8ecd-e0cb16742036 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:24.667147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f04b73-5150-49aa-8ecd-e0cb16742036 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:24.669787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:24.678110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f04b73-5150-49aa-8ecd-e0cb16742036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:24.680937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 550/1088] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b7f04b73-5150-49aa-8ecd-e0cb16742036 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:24.686461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:24.687571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:24.688426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 539/1089] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:53:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:24.693536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388edff-8a2c-42ca-aee0-97150f56f802 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:24.696721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:24.704964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f04b73-5150-49aa-8ecd-e0cb16742036 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:24.794110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2388edff-8a2c-42ca-aee0-97150f56f802 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:24.803217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f04b73-5150-49aa-8ecd-e0cb16742036 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:24.827357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e7336-ae4b-4616-8b1c-920dc8458fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:24.828018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e7336-ae4b-4616-8b1c-920dc8458fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:24.828303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e7336-ae4b-4616-8b1c-920dc8458fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:24.828580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894e7336-ae4b-4616-8b1c-920dc8458fa4 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:24.829322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e7336-ae4b-4616-8b1c-920dc8458fa4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:24.829634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e7336-ae4b-4616-8b1c-920dc8458fa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:24.831036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894e7336-ae4b-4616-8b1c-920dc8458fa4 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:24.836114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb67c5-fd4a-42fa-9699-4572d98ed14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:24.836912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb67c5-fd4a-42fa-9699-4572d98ed14a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:24.837197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb67c5-fd4a-42fa-9699-4572d98ed14a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:24.838023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb67c5-fd4a-42fa-9699-4572d98ed14a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:24.839696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 551/1090] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:53:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:24.845423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e7336-ae4b-4616-8b1c-920dc8458fa4 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:24.850513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fb67c5-fd4a-42fa-9699-4572d98ed14a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-99fb67c5-fd4a-42fa-9699-4572d98ed14a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:24.856824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:24.857923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:24.859047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 540/1091] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Jun 26 02:53:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:24.863871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb67c5-fd4a-42fa-9699-4572d98ed14a None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:24.913179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:24.920081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:24.948185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894e7336-ae4b-4616-8b1c-920dc8458fa4 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:24.964331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fb67c5-fd4a-42fa-9699-4572d98ed14a None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:24.980451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718206ea-0f8b-4db0-af73-cb217f039553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:24.981318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718206ea-0f8b-4db0-af73-cb217f039553 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:24.981595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718206ea-0f8b-4db0-af73-cb217f039553 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:24.981864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718206ea-0f8b-4db0-af73-cb217f039553 None admin] PATH_INFO: `/v3/users/50134745807f42f1ab662f7d2fea69c7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:24.983062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718206ea-0f8b-4db0-af73-cb217f039553 None admin] RBAC: Authorizing `identity:get_user(user_id=50134745807f42f1ab662f7d2fea69c7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:24.984527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718206ea-0f8b-4db0-af73-cb217f039553 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:24.986187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 552/1092] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:24 2022] GET /identity/v3/users/50134745807f42f1ab662f7d2fea69c7 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:24.991876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718206ea-0f8b-4db0-af73-cb217f039553 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:25.001151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb8df3-2542-4d01-aef7-337bf13de33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:25.001811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb8df3-2542-4d01-aef7-337bf13de33f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:25.002114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb8df3-2542-4d01-aef7-337bf13de33f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:25.002392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdb8df3-2542-4d01-aef7-337bf13de33f None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:25.003134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb8df3-2542-4d01-aef7-337bf13de33f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:25.003438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb8df3-2542-4d01-aef7-337bf13de33f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:25.004776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdb8df3-2542-4d01-aef7-337bf13de33f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:25.013124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 541/1093] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Jun 26 02:53:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:25.018623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb8df3-2542-4d01-aef7-337bf13de33f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:25.094981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718206ea-0f8b-4db0-af73-cb217f039553 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:25.115444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdb8df3-2542-4d01-aef7-337bf13de33f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:25.128831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06626852-334c-406f-8eef-43c589221a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:25.129487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06626852-334c-406f-8eef-43c589221a2e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:25.129773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06626852-334c-406f-8eef-43c589221a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:25.130069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06626852-334c-406f-8eef-43c589221a2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:25.132691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:25.132938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:25.150976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e02646-9235-4704-b8f7-e329c00edbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:25.151757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02646-9235-4704-b8f7-e329c00edbfc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:25.152042 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02646-9235-4704-b8f7-e329c00edbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:25.152311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e02646-9235-4704-b8f7-e329c00edbfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:25.162633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e02646-9235-4704-b8f7-e329c00edbfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:25.164060 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e02646-9235-4704-b8f7-e329c00edbfc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-19e02646-9235-4704-b8f7-e329c00edbfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:25.180642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:25.181999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:25.182915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:25.182915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:25.182915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:25.182915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:25.182915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 542/1094] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:25.188166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e02646-9235-4704-b8f7-e329c00edbfc None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:25.200911 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06626852-334c-406f-8eef-43c589221a2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:25.202391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06626852-334c-406f-8eef-43c589221a2e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:25.283503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e02646-9235-4704-b8f7-e329c00edbfc None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:25.298799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 553/1095] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 26 02:53:25.304587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06626852-334c-406f-8eef-43c589221a2e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:25.316201 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08246a5-9cb3-4abe-a3ce-292341832579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:25.316854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08246a5-9cb3-4abe-a3ce-292341832579 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:25.317137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08246a5-9cb3-4abe-a3ce-292341832579 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:25.317413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08246a5-9cb3-4abe-a3ce-292341832579 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:25.318158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08246a5-9cb3-4abe-a3ce-292341832579 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:25.318466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08246a5-9cb3-4abe-a3ce-292341832579 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:25.319926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08246a5-9cb3-4abe-a3ce-292341832579 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:25.331357 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 543/1096] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:25.336683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08246a5-9cb3-4abe-a3ce-292341832579 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:25.337547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:25.366005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:25.403432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06626852-334c-406f-8eef-43c589221a2e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:25.435443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08246a5-9cb3-4abe-a3ce-292341832579 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:25.440909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef273937-598a-439d-95a6-b51def520ecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:25.441774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef273937-598a-439d-95a6-b51def520ecc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:25.442068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef273937-598a-439d-95a6-b51def520ecc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:25.442354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef273937-598a-439d-95a6-b51def520ecc None admin] PATH_INFO: `/v3/projects/628baa9ce0074b99ba0dd86a7223ae81` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:25.443534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef273937-598a-439d-95a6-b51def520ecc None admin] RBAC: Authorizing `identity:get_project(project_id=628baa9ce0074b99ba0dd86a7223ae81)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:25.444956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef273937-598a-439d-95a6-b51def520ecc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:25.446733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 554/1097] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:53:25 2022] GET /identity/v3/projects/628baa9ce0074b99ba0dd86a7223ae81 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:25.452508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef273937-598a-439d-95a6-b51def520ecc None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:25.468662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbcf67-dc98-4125-8232-11376e1cbbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:25.469444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbcf67-dc98-4125-8232-11376e1cbbdd None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:25.469725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbcf67-dc98-4125-8232-11376e1cbbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:25.469997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdbcf67-dc98-4125-8232-11376e1cbbdd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:25.477001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbcf67-dc98-4125-8232-11376e1cbbdd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:25.478950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdbcf67-dc98-4125-8232-11376e1cbbdd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-9bdbcf67-dc98-4125-8232-11376e1cbbdd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:53:25.493267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:25.494855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:25.495792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 544/1098] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Jun 26 02:53:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:25.500913 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbcf67-dc98-4125-8232-11376e1cbbdd None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:25.550901 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef273937-598a-439d-95a6-b51def520ecc None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:25.552400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:25.584859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec95517-2eae-4072-8b7d-9f7fec526108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:25.585503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec95517-2eae-4072-8b7d-9f7fec526108 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:25.585784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec95517-2eae-4072-8b7d-9f7fec526108 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:25.586082 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec95517-2eae-4072-8b7d-9f7fec526108 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:25.586904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec95517-2eae-4072-8b7d-9f7fec526108 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=628baa9ce0074b99ba0dd86a7223ae81, user.id=50134745807f42f1ab662f7d2fea69c7) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:25.587235 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec95517-2eae-4072-8b7d-9f7fec526108 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:25.587987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:25.588647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec95517-2eae-4072-8b7d-9f7fec526108 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:25.594783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 555/1099] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:53:25 2022] GET /identity/v3/role_assignments?user.id=50134745807f42f1ab662f7d2fea69c7&scope.project.id=628baa9ce0074b99ba0dd86a7223ae81&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:25.598090 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdbcf67-dc98-4125-8232-11376e1cbbdd None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:25.636500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e909ab6-80bc-498b-841e-8c021eb9c39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:25.637185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e909ab6-80bc-498b-841e-8c021eb9c39c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:25.637470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e909ab6-80bc-498b-841e-8c021eb9c39c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:25.637746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e909ab6-80bc-498b-841e-8c021eb9c39c None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:25.638633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e909ab6-80bc-498b-841e-8c021eb9c39c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:25.638944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e909ab6-80bc-498b-841e-8c021eb9c39c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:25.640408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e909ab6-80bc-498b-841e-8c021eb9c39c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:25.660214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 545/1100] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Jun 26 02:53:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:25.666723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec95517-2eae-4072-8b7d-9f7fec526108 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:25.769807 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec95517-2eae-4072-8b7d-9f7fec526108 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:25.803879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8109283-71eb-42ea-a9b7-5f2bb603775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:25.804527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8109283-71eb-42ea-a9b7-5f2bb603775b None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:25.804810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8109283-71eb-42ea-a9b7-5f2bb603775b None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:25.805087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8109283-71eb-42ea-a9b7-5f2bb603775b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:25.837774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:25.874474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8109283-71eb-42ea-a9b7-5f2bb603775b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:25.875923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8109283-71eb-42ea-a9b7-5f2bb603775b None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:25.971536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 556/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:53:25.977165 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e909ab6-80bc-498b-841e-8c021eb9c39c None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:26.073934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e909ab6-80bc-498b-841e-8c021eb9c39c None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:26.107555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c75956-57c7-426f-a1ed-335079f98063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:26.108348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c75956-57c7-426f-a1ed-335079f98063 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:26.108630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c75956-57c7-426f-a1ed-335079f98063 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:26.108905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c75956-57c7-426f-a1ed-335079f98063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:26.118831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c75956-57c7-426f-a1ed-335079f98063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:26.120256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c75956-57c7-426f-a1ed-335079f98063 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:26.128708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-58c75956-57c7-426f-a1ed-335079f98063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:26.138993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:26.139946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:26.140830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:53:26.140830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:26.140830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:53:26.140830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:26.140830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 546/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:26.145958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8109283-71eb-42ea-a9b7-5f2bb603775b None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:26.208500 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:26.242186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8109283-71eb-42ea-a9b7-5f2bb603775b None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:26.276490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf5014-fc29-4394-8f42-1508d4f158a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:26.277145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf5014-fc29-4394-8f42-1508d4f158a2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:26.277426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf5014-fc29-4394-8f42-1508d4f158a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:26.277703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecf5014-fc29-4394-8f42-1508d4f158a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:26.278455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf5014-fc29-4394-8f42-1508d4f158a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:26.278760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf5014-fc29-4394-8f42-1508d4f158a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:26.280114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecf5014-fc29-4394-8f42-1508d4f158a2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:26.291092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 557/1103] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:26.296785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c75956-57c7-426f-a1ed-335079f98063 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:26.392960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c75956-57c7-426f-a1ed-335079f98063 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:26.425970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad672d1-c54a-430d-96cd-20f412662d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:26.426775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad672d1-c54a-430d-96cd-20f412662d26 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:26.427060 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad672d1-c54a-430d-96cd-20f412662d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:26.427336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad672d1-c54a-430d-96cd-20f412662d26 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:26.434366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad672d1-c54a-430d-96cd-20f412662d26 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:26.435792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad672d1-c54a-430d-96cd-20f412662d26 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-1ad672d1-c54a-430d-96cd-20f412662d26 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:53:26.449632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:26.450559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:26.451452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:53:26.451452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:53:26.451452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:53:26.451452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:53:26.451452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:53:26.451452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:53:26.451452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:53:26.451452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:26.451452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 547/1104] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:53:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:53:26.458117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf5014-fc29-4394-8f42-1508d4f158a2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:26.554412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecf5014-fc29-4394-8f42-1508d4f158a2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:26.579492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:26.588374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d5b70a-e905-4902-a343-730a7b6527ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:26.589029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5b70a-e905-4902-a343-730a7b6527ab None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:26.589316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5b70a-e905-4902-a343-730a7b6527ab None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:26.589594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d5b70a-e905-4902-a343-730a7b6527ab None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:26.590417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5b70a-e905-4902-a343-730a7b6527ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:26.590731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5b70a-e905-4902-a343-730a7b6527ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:26.592155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d5b70a-e905-4902-a343-730a7b6527ab None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:26.604371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 558/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:26.610329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad672d1-c54a-430d-96cd-20f412662d26 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:26.668609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:26.709687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad672d1-c54a-430d-96cd-20f412662d26 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:26.746122 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7870b74-b5b5-4809-8438-9e8b52a32857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:26.746782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7870b74-b5b5-4809-8438-9e8b52a32857 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:26.747071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7870b74-b5b5-4809-8438-9e8b52a32857 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:26.747346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7870b74-b5b5-4809-8438-9e8b52a32857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:26.748181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7870b74-b5b5-4809-8438-9e8b52a32857 None admin] RBAC: Adding query filter params (role.id=d34bc405b0c04033a57dc7dae296fdee, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=9ee0150dfb3f42709029e42d0fd9105d) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:26.748495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7870b74-b5b5-4809-8438-9e8b52a32857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:26.750204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7870b74-b5b5-4809-8438-9e8b52a32857 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:26.756029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 548/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:53:26 2022] GET /identity/v3/role_assignments?user.id=9ee0150dfb3f42709029e42d0fd9105d&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=d34bc405b0c04033a57dc7dae296fdee => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:27.626844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f543b7-d60b-49e4-92b2-fbbab62a27f0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:27.627388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f543b7-d60b-49e4-92b2-fbbab62a27f0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:27.627388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f543b7-d60b-49e4-92b2-fbbab62a27f0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:27.628793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 559/1107] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:27.633794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c425-d75a-4477-ab80-b6e3e976f166 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:27.634245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c425-d75a-4477-ab80-b6e3e976f166 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:27.634383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89c425-d75a-4477-ab80-b6e3e976f166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:27.650658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2a89c425-d75a-4477-ab80-b6e3e976f166 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:27.651835 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2a89c425-d75a-4477-ab80-b6e3e976f166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:27.715537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 549/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:27.740046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f543b7-d60b-49e4-92b2-fbbab62a27f0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:27.842455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f543b7-d60b-49e4-92b2-fbbab62a27f0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:27.875394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c30b3aa-cd18-41c7-a235-48fde33d36ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:27.876171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b3aa-cd18-41c7-a235-48fde33d36ba None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:27.876448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b3aa-cd18-41c7-a235-48fde33d36ba None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:27.876716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c30b3aa-cd18-41c7-a235-48fde33d36ba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:27.877921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30b3aa-cd18-41c7-a235-48fde33d36ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:27.879305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c30b3aa-cd18-41c7-a235-48fde33d36ba None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:27.880921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 560/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:27.887098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c425-d75a-4477-ab80-b6e3e976f166 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:27.941667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:27.984304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89c425-d75a-4477-ab80-b6e3e976f166 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:28.018343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cddfc68-be12-43ff-9c9d-0c9021234fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:28.019008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddfc68-be12-43ff-9c9d-0c9021234fe1 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:28.019292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddfc68-be12-43ff-9c9d-0c9021234fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:28.019564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cddfc68-be12-43ff-9c9d-0c9021234fe1 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:28.020278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddfc68-be12-43ff-9c9d-0c9021234fe1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:28.021700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cddfc68-be12-43ff-9c9d-0c9021234fe1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:28.040261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 550/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:53:27 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:53:30.074936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf6e8c-e7f6-43c0-bff6-104ba72a3c7a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:30.075454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf6e8c-e7f6-43c0-bff6-104ba72a3c7a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:30.075544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bf6e8c-e7f6-43c0-bff6-104ba72a3c7a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:30.076880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 561/1111] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:30.082006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980bd56-7da2-4ed1-a5dd-4df2b1728055 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:30.082562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980bd56-7da2-4ed1-a5dd-4df2b1728055 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:30.082655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980bd56-7da2-4ed1-a5dd-4df2b1728055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:30.098710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f980bd56-7da2-4ed1-a5dd-4df2b1728055 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:30.099878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f980bd56-7da2-4ed1-a5dd-4df2b1728055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:30.164075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 551/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:30.188166 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf6e8c-e7f6-43c0-bff6-104ba72a3c7a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:30.243248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:30.287293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bf6e8c-e7f6-43c0-bff6-104ba72a3c7a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:30.319386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fc0f6-4251-4455-841e-0a4c83161ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:30.320223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fc0f6-4251-4455-841e-0a4c83161ddf None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:30.320502 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fc0f6-4251-4455-841e-0a4c83161ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:30.320774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915fc0f6-4251-4455-841e-0a4c83161ddf None admin] PATH_INFO: `/v3/roles/4f31bdaad0194bbd8fab39617059d14a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:30.322237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fc0f6-4251-4455-841e-0a4c83161ddf None admin] RBAC: Authorizing `identity:get_role(role_id=4f31bdaad0194bbd8fab39617059d14a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:30.323571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915fc0f6-4251-4455-841e-0a4c83161ddf None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:30.324929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 562/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:30 2022] GET /identity/v3/roles/4f31bdaad0194bbd8fab39617059d14a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:30.330446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980bd56-7da2-4ed1-a5dd-4df2b1728055 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:30.370269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:30.425734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980bd56-7da2-4ed1-a5dd-4df2b1728055 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:30.457989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829ffc-4418-4b2a-9ad1-27cc7b2bd8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:30.458870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829ffc-4418-4b2a-9ad1-27cc7b2bd8c8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:30.459150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829ffc-4418-4b2a-9ad1-27cc7b2bd8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:30.459422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82829ffc-4418-4b2a-9ad1-27cc7b2bd8c8 None admin] PATH_INFO: `/v3/users/14605b3c2a53453195f6def210a6e8a8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:30.472346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82829ffc-4418-4b2a-9ad1-27cc7b2bd8c8 None admin] RBAC: Authorizing `identity:get_user(user_id=14605b3c2a53453195f6def210a6e8a8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:30.473749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82829ffc-4418-4b2a-9ad1-27cc7b2bd8c8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:30.475421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 552/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:30 2022] GET /identity/v3/users/14605b3c2a53453195f6def210a6e8a8 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:30.481846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fc0f6-4251-4455-841e-0a4c83161ddf None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:30.577194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915fc0f6-4251-4455-841e-0a4c83161ddf None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:30.599242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:30.610917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b12053-b2e9-4a94-8822-da5f5dc6ec87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:30.611555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b12053-b2e9-4a94-8822-da5f5dc6ec87 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:30.611838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b12053-b2e9-4a94-8822-da5f5dc6ec87 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:30.612114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b12053-b2e9-4a94-8822-da5f5dc6ec87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:30.676549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b12053-b2e9-4a94-8822-da5f5dc6ec87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:30.677909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b12053-b2e9-4a94-8822-da5f5dc6ec87 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:30.772458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 563/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:53:30.778905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829ffc-4418-4b2a-9ad1-27cc7b2bd8c8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:30.869879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:30.873929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82829ffc-4418-4b2a-9ad1-27cc7b2bd8c8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:30.908293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b1364e-a8e4-4883-9a16-895fd4672bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:30.909143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1364e-a8e4-4883-9a16-895fd4672bef None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:30.909426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1364e-a8e4-4883-9a16-895fd4672bef None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:30.909699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b1364e-a8e4-4883-9a16-895fd4672bef None admin] PATH_INFO: `/v3/projects/628baa9ce0074b99ba0dd86a7223ae81` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:30.910882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1364e-a8e4-4883-9a16-895fd4672bef None admin] RBAC: Authorizing `identity:get_project(project_id=628baa9ce0074b99ba0dd86a7223ae81)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:30.912292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b1364e-a8e4-4883-9a16-895fd4672bef None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:30.914023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 553/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:53:30 2022] GET /identity/v3/projects/628baa9ce0074b99ba0dd86a7223ae81 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:30.920061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b12053-b2e9-4a94-8822-da5f5dc6ec87 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:30.943330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:31.016222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b12053-b2e9-4a94-8822-da5f5dc6ec87 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:31.048254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a235a-ca88-4b2e-9844-9d10375d5883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:31.048895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a235a-ca88-4b2e-9844-9d10375d5883 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:31.049178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a235a-ca88-4b2e-9844-9d10375d5883 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:31.049454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a235a-ca88-4b2e-9844-9d10375d5883 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:31.050296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a235a-ca88-4b2e-9844-9d10375d5883 None admin] RBAC: Adding query filter params (role.id=4f31bdaad0194bbd8fab39617059d14a, scope.project.id=628baa9ce0074b99ba0dd86a7223ae81, user.id=14605b3c2a53453195f6def210a6e8a8) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:31.050605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a235a-ca88-4b2e-9844-9d10375d5883 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:31.052014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a235a-ca88-4b2e-9844-9d10375d5883 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:31.057863 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 564/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:53:30 2022] GET /identity/v3/role_assignments?user.id=14605b3c2a53453195f6def210a6e8a8&scope.project.id=628baa9ce0074b99ba0dd86a7223ae81&role.id=4f31bdaad0194bbd8fab39617059d14a => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:33.044158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6717971-f3e7-449a-83b0-6506a798857a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:33.044673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6717971-f3e7-449a-83b0-6506a798857a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:33.044780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6717971-f3e7-449a-83b0-6506a798857a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:33.046148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 554/1118] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:33.051987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ffa34-7877-48fd-8cd4-0e766b62b082 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:33.052395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ffa34-7877-48fd-8cd4-0e766b62b082 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:33.052536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ffa34-7877-48fd-8cd4-0e766b62b082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:33.069244 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-979ffa34-7877-48fd-8cd4-0e766b62b082 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:33.070459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-979ffa34-7877-48fd-8cd4-0e766b62b082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:33.133990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 565/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:33.157388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6717971-f3e7-449a-83b0-6506a798857a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:33.255289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6717971-f3e7-449a-83b0-6506a798857a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:33.287680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f88b3-16b6-4234-b486-b4fc42e34c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:33.288522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f88b3-16b6-4234-b486-b4fc42e34c56 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:33.288811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f88b3-16b6-4234-b486-b4fc42e34c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:33.289089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f88b3-16b6-4234-b486-b4fc42e34c56 None admin] PATH_INFO: `/v3/roles/4f31bdaad0194bbd8fab39617059d14a` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:33.291166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f88b3-16b6-4234-b486-b4fc42e34c56 None admin] RBAC: Authorizing `identity:get_role(role_id=4f31bdaad0194bbd8fab39617059d14a)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:33.292522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f88b3-16b6-4234-b486-b4fc42e34c56 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:33.293890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 555/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:33 2022] GET /identity/v3/roles/4f31bdaad0194bbd8fab39617059d14a => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:33.299614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ffa34-7877-48fd-8cd4-0e766b62b082 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:33.316180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:33.402413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ffa34-7877-48fd-8cd4-0e766b62b082 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:33.434674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93520d4-bfa2-4ce7-894d-9e3cceb97a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:33.435525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93520d4-bfa2-4ce7-894d-9e3cceb97a61 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:33.435801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93520d4-bfa2-4ce7-894d-9e3cceb97a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:33.436067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93520d4-bfa2-4ce7-894d-9e3cceb97a61 None admin] PATH_INFO: `/v3/users/14605b3c2a53453195f6def210a6e8a8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:33.437221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93520d4-bfa2-4ce7-894d-9e3cceb97a61 None admin] RBAC: Authorizing `identity:get_user(user_id=14605b3c2a53453195f6def210a6e8a8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:33.438662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93520d4-bfa2-4ce7-894d-9e3cceb97a61 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:33.440298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 566/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:33 2022] GET /identity/v3/users/14605b3c2a53453195f6def210a6e8a8 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:33.446195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f88b3-16b6-4234-b486-b4fc42e34c56 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:33.479803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:33.543448 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f88b3-16b6-4234-b486-b4fc42e34c56 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:33.575659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98280bdd-6dae-4a91-b9f6-abdc791801ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:33.576495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280bdd-6dae-4a91-b9f6-abdc791801ff None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:33.576780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280bdd-6dae-4a91-b9f6-abdc791801ff None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:33.577055 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98280bdd-6dae-4a91-b9f6-abdc791801ff None admin] PATH_INFO: `/v3/projects/628baa9ce0074b99ba0dd86a7223ae81` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:33.578251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98280bdd-6dae-4a91-b9f6-abdc791801ff None admin] RBAC: Authorizing `identity:get_project(project_id=628baa9ce0074b99ba0dd86a7223ae81)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:33.579661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98280bdd-6dae-4a91-b9f6-abdc791801ff None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:33.581350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 556/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:53:33 2022] GET /identity/v3/projects/628baa9ce0074b99ba0dd86a7223ae81 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:33.587655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93520d4-bfa2-4ce7-894d-9e3cceb97a61 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:33.668780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:33.684188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93520d4-bfa2-4ce7-894d-9e3cceb97a61 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:33.717334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d79bc-aa4e-40f5-b0a6-842607fa257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:33.718229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d79bc-aa4e-40f5-b0a6-842607fa257f None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:33.718509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d79bc-aa4e-40f5-b0a6-842607fa257f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:33.718780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d79bc-aa4e-40f5-b0a6-842607fa257f None admin] PATH_INFO: `/v3/projects/628baa9ce0074b99ba0dd86a7223ae81/users/14605b3c2a53453195f6def210a6e8a8/roles/4f31bdaad0194bbd8fab39617059d14a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:33.720917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d79bc-aa4e-40f5-b0a6-842607fa257f None admin] RBAC: Authorizing `identity:create_grant(project_id=628baa9ce0074b99ba0dd86a7223ae81, user_id=14605b3c2a53453195f6def210a6e8a8, role_id=4f31bdaad0194bbd8fab39617059d14a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:33.722349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d79bc-aa4e-40f5-b0a6-842607fa257f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:33.734265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 567/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:53:33 2022] PUT /identity/v3/projects/628baa9ce0074b99ba0dd86a7223ae81/users/14605b3c2a53453195f6def210a6e8a8/roles/4f31bdaad0194bbd8fab39617059d14a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:53:35.717900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861172b-d9c5-46d5-967d-117620fed611 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:35.718459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861172b-d9c5-46d5-967d-117620fed611 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:35.718545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861172b-d9c5-46d5-967d-117620fed611 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:35.719905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 557/1124] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:35.725108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461be571-36d6-4a91-b1e7-8bc94be07ce2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:35.725513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461be571-36d6-4a91-b1e7-8bc94be07ce2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:35.725675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461be571-36d6-4a91-b1e7-8bc94be07ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:35.742659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-461be571-36d6-4a91-b1e7-8bc94be07ce2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:35.743848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-461be571-36d6-4a91-b1e7-8bc94be07ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:35.808583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 568/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:35.832166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861172b-d9c5-46d5-967d-117620fed611 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:35.932999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861172b-d9c5-46d5-967d-117620fed611 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:35.934523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:35.967445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee100035-5e2d-4e17-bde8-f1207b433fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:35.968330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee100035-5e2d-4e17-bde8-f1207b433fcd None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:35.968568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee100035-5e2d-4e17-bde8-f1207b433fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:35.968844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee100035-5e2d-4e17-bde8-f1207b433fcd None admin] PATH_INFO: `/v3/roles/4f31bdaad0194bbd8fab39617059d14a` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:35.970324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee100035-5e2d-4e17-bde8-f1207b433fcd None admin] RBAC: Authorizing `identity:get_role(role_id=4f31bdaad0194bbd8fab39617059d14a)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:35.971683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee100035-5e2d-4e17-bde8-f1207b433fcd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:35.973082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 558/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:35 2022] GET /identity/v3/roles/4f31bdaad0194bbd8fab39617059d14a => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:35.978907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461be571-36d6-4a91-b1e7-8bc94be07ce2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:36.051722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:36.076686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461be571-36d6-4a91-b1e7-8bc94be07ce2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:36.109981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45506bd-a94a-4042-812b-80b048c65081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:36.110864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45506bd-a94a-4042-812b-80b048c65081 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:36.111148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45506bd-a94a-4042-812b-80b048c65081 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:36.111424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45506bd-a94a-4042-812b-80b048c65081 None admin] PATH_INFO: `/v3/users/14605b3c2a53453195f6def210a6e8a8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:36.112588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45506bd-a94a-4042-812b-80b048c65081 None admin] RBAC: Authorizing `identity:get_user(user_id=14605b3c2a53453195f6def210a6e8a8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:36.114008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45506bd-a94a-4042-812b-80b048c65081 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:36.115666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 569/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:35 2022] GET /identity/v3/users/14605b3c2a53453195f6def210a6e8a8 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:36.121567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee100035-5e2d-4e17-bde8-f1207b433fcd None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:36.216451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee100035-5e2d-4e17-bde8-f1207b433fcd None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:36.248453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4e081-2b79-46b9-bee1-782bbc584a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:36.249278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4e081-2b79-46b9-bee1-782bbc584a04 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:36.249568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4e081-2b79-46b9-bee1-782bbc584a04 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:36.249838 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4e081-2b79-46b9-bee1-782bbc584a04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:36.286357 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:36.314267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4e081-2b79-46b9-bee1-782bbc584a04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:36.315708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4e081-2b79-46b9-bee1-782bbc584a04 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:36.408983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 559/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:53:36.414822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45506bd-a94a-4042-812b-80b048c65081 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:36.511997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45506bd-a94a-4042-812b-80b048c65081 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:36.544598 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770f4798-ceae-4af2-8318-4848667b9d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:36.545437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4798-ceae-4af2-8318-4848667b9d3f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:36.545717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4798-ceae-4af2-8318-4848667b9d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:36.545989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770f4798-ceae-4af2-8318-4848667b9d3f None admin] PATH_INFO: `/v3/projects/628baa9ce0074b99ba0dd86a7223ae81` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:36.547192 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f4798-ceae-4af2-8318-4848667b9d3f None admin] RBAC: Authorizing `identity:get_project(project_id=628baa9ce0074b99ba0dd86a7223ae81)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:36.548608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770f4798-ceae-4af2-8318-4848667b9d3f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:36.550346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 570/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:53:36 2022] GET /identity/v3/projects/628baa9ce0074b99ba0dd86a7223ae81 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:36.556849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4e081-2b79-46b9-bee1-782bbc584a04 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:36.622875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:36.653128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4e081-2b79-46b9-bee1-782bbc584a04 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:36.686458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a85768-6091-425c-8154-501eb8df175f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:36.687103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a85768-6091-425c-8154-501eb8df175f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:36.687385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a85768-6091-425c-8154-501eb8df175f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:36.687659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a85768-6091-425c-8154-501eb8df175f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:36.688483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a85768-6091-425c-8154-501eb8df175f None admin] RBAC: Adding query filter params (role.id=4f31bdaad0194bbd8fab39617059d14a, scope.project.id=628baa9ce0074b99ba0dd86a7223ae81, user.id=14605b3c2a53453195f6def210a6e8a8) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:36.688791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a85768-6091-425c-8154-501eb8df175f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:36.690217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a85768-6091-425c-8154-501eb8df175f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:36.695988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 560/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:53:36 2022] GET /identity/v3/role_assignments?user.id=14605b3c2a53453195f6def210a6e8a8&scope.project.id=628baa9ce0074b99ba0dd86a7223ae81&role.id=4f31bdaad0194bbd8fab39617059d14a => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:38.704797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5387f5-579a-4e8c-804b-75ab421a115b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:38.705343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5387f5-579a-4e8c-804b-75ab421a115b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:38.705343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5387f5-579a-4e8c-804b-75ab421a115b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:38.706759 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 571/1131] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:38.712397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8272f-3a00-4f95-b9cf-2ad42425e1db None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:38.712803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8272f-3a00-4f95-b9cf-2ad42425e1db None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:38.712959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a8272f-3a00-4f95-b9cf-2ad42425e1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:38.729582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-42a8272f-3a00-4f95-b9cf-2ad42425e1db None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:38.730783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-42a8272f-3a00-4f95-b9cf-2ad42425e1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:38.794116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 561/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:38.817035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5387f5-579a-4e8c-804b-75ab421a115b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:38.829474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:38.917485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5387f5-579a-4e8c-804b-75ab421a115b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:38.949949 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52868c-ef1d-4859-b339-d2b34bc80381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:38.950761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52868c-ef1d-4859-b339-d2b34bc80381 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:38.951045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52868c-ef1d-4859-b339-d2b34bc80381 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:38.951321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52868c-ef1d-4859-b339-d2b34bc80381 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:38.952473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52868c-ef1d-4859-b339-d2b34bc80381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:38.953880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52868c-ef1d-4859-b339-d2b34bc80381 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:38.955614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 572/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:38.961332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8272f-3a00-4f95-b9cf-2ad42425e1db None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:39.020192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:39.056597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a8272f-3a00-4f95-b9cf-2ad42425e1db None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:39.090133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d178d3e-e714-4f83-8e60-e801b9d22484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:39.090778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d178d3e-e714-4f83-8e60-e801b9d22484 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:39.091061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d178d3e-e714-4f83-8e60-e801b9d22484 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:39.091336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d178d3e-e714-4f83-8e60-e801b9d22484 None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:39.092045 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d178d3e-e714-4f83-8e60-e801b9d22484 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:39.093452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d178d3e-e714-4f83-8e60-e801b9d22484 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:39.116961 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 562/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:53:38 2022] POST /identity/v3/projects => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:53:41.161777 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb6e2a-89af-49fe-b39a-6e048bf5d0d0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:41.162415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb6e2a-89af-49fe-b39a-6e048bf5d0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:41.162415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabb6e2a-89af-49fe-b39a-6e048bf5d0d0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:41.163765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 573/1135] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:41.169446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7162c2f-8642-4046-a8b5-e4cbbc99709e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:41.169856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7162c2f-8642-4046-a8b5-e4cbbc99709e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:41.170011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7162c2f-8642-4046-a8b5-e4cbbc99709e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:41.187040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b7162c2f-8642-4046-a8b5-e4cbbc99709e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:41.188207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b7162c2f-8642-4046-a8b5-e4cbbc99709e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:41.251435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 563/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:41.275895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb6e2a-89af-49fe-b39a-6e048bf5d0d0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:41.352194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:41.375399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabb6e2a-89af-49fe-b39a-6e048bf5d0d0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:41.408023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102f8087-2a81-4bf6-aad3-8916d2444506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:41.408790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f8087-2a81-4bf6-aad3-8916d2444506 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:41.409070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f8087-2a81-4bf6-aad3-8916d2444506 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:41.409341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102f8087-2a81-4bf6-aad3-8916d2444506 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:41.410567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f8087-2a81-4bf6-aad3-8916d2444506 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:41.412026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102f8087-2a81-4bf6-aad3-8916d2444506 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:41.413593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 574/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:53:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:41.419898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7162c2f-8642-4046-a8b5-e4cbbc99709e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:41.463922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:41.515306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7162c2f-8642-4046-a8b5-e4cbbc99709e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:41.548353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6a3174-6a82-452f-9d6e-1e9632ccdcd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:41.549007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3174-6a82-452f-9d6e-1e9632ccdcd5 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:41.549291 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3174-6a82-452f-9d6e-1e9632ccdcd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:41.549568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6a3174-6a82-452f-9d6e-1e9632ccdcd5 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:41.550412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a3174-6a82-452f-9d6e-1e9632ccdcd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:41.551828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6a3174-6a82-452f-9d6e-1e9632ccdcd5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:41.570314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 564/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:53:41 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:53:43.608104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f50c9f-683e-422e-bffa-b15ee4e99727 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:43.608660 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f50c9f-683e-422e-bffa-b15ee4e99727 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:43.608660 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f50c9f-683e-422e-bffa-b15ee4e99727 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:43.610172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 575/1139] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:43.615203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa73f74-847e-4bd3-af2a-3296277a7f52 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:43.615621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa73f74-847e-4bd3-af2a-3296277a7f52 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:43.615766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa73f74-847e-4bd3-af2a-3296277a7f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:43.632260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3fa73f74-847e-4bd3-af2a-3296277a7f52 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:43.633437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3fa73f74-847e-4bd3-af2a-3296277a7f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:43.712792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 565/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:43.739173 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f50c9f-683e-422e-bffa-b15ee4e99727 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:43.836414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f50c9f-683e-422e-bffa-b15ee4e99727 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:43.869444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103aaab-b62e-4d3f-a81d-f2fd951196f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:43.870265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103aaab-b62e-4d3f-a81d-f2fd951196f5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:43.870551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103aaab-b62e-4d3f-a81d-f2fd951196f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:43.870827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2103aaab-b62e-4d3f-a81d-f2fd951196f5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:43.880922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2103aaab-b62e-4d3f-a81d-f2fd951196f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-2103aaab-b62e-4d3f-a81d-f2fd951196f5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:43.889160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:43.890335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:43.891482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:43.894343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 576/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:53:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:43.896206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa73f74-847e-4bd3-af2a-3296277a7f52 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:43.925150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:43.991773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa73f74-847e-4bd3-af2a-3296277a7f52 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:44.024796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebce903-fb20-4dc1-8947-64079d4de009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:44.025445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebce903-fb20-4dc1-8947-64079d4de009 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:44.025726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebce903-fb20-4dc1-8947-64079d4de009 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:44.025997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebce903-fb20-4dc1-8947-64079d4de009 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:44.026762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebce903-fb20-4dc1-8947-64079d4de009 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:44.027065 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebce903-fb20-4dc1-8947-64079d4de009 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:44.028414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebce903-fb20-4dc1-8947-64079d4de009 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:44.036133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 566/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:53:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:44.042366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103aaab-b62e-4d3f-a81d-f2fd951196f5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:44.051331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:44.138459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2103aaab-b62e-4d3f-a81d-f2fd951196f5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:44.170156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb72483-9171-49f3-87a2-accb762ba421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:44.171054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb72483-9171-49f3-87a2-accb762ba421 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:44.171331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb72483-9171-49f3-87a2-accb762ba421 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:44.171603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb72483-9171-49f3-87a2-accb762ba421 None admin] PATH_INFO: `/v3/users/03cc61e3ccb34ecca1eb16cd8199b9e1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:44.184817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb72483-9171-49f3-87a2-accb762ba421 None admin] RBAC: Authorizing `identity:get_user(user_id=03cc61e3ccb34ecca1eb16cd8199b9e1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:44.186269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb72483-9171-49f3-87a2-accb762ba421 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:44.187925 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 577/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:44 2022] GET /identity/v3/users/03cc61e3ccb34ecca1eb16cd8199b9e1 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:44.193765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebce903-fb20-4dc1-8947-64079d4de009 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:44.283056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:44.290351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebce903-fb20-4dc1-8947-64079d4de009 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:44.322478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213534a-b9d7-4fdb-bb14-ce28c0dcf0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:44.323112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213534a-b9d7-4fdb-bb14-ce28c0dcf0be None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:44.323399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213534a-b9d7-4fdb-bb14-ce28c0dcf0be None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:44.323679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1213534a-b9d7-4fdb-bb14-ce28c0dcf0be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:44.387934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213534a-b9d7-4fdb-bb14-ce28c0dcf0be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:44.389411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1213534a-b9d7-4fdb-bb14-ce28c0dcf0be None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:44.476628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:44.484500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 567/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:53:44.490133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb72483-9171-49f3-87a2-accb762ba421 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:44.550881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:44.586568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb72483-9171-49f3-87a2-accb762ba421 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:44.619633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d2b1fd-de86-4ab4-a783-981bba076181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:44.620485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2b1fd-de86-4ab4-a783-981bba076181 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:44.620767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2b1fd-de86-4ab4-a783-981bba076181 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:44.621041 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d2b1fd-de86-4ab4-a783-981bba076181 None admin] PATH_INFO: `/v3/projects/3df3fc4e561247979a49d0b052eaee73` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:44.622238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2b1fd-de86-4ab4-a783-981bba076181 None admin] RBAC: Authorizing `identity:get_project(project_id=3df3fc4e561247979a49d0b052eaee73)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:44.623656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d2b1fd-de86-4ab4-a783-981bba076181 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:44.625609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 578/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:53:44 2022] GET /identity/v3/projects/3df3fc4e561247979a49d0b052eaee73 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:44.631475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213534a-b9d7-4fdb-bb14-ce28c0dcf0be None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:44.725752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1213534a-b9d7-4fdb-bb14-ce28c0dcf0be None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:44.757932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fbd1e-ef52-41ce-9971-eee52188606e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:44.758613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fbd1e-ef52-41ce-9971-eee52188606e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:44.758902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fbd1e-ef52-41ce-9971-eee52188606e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:44.759173 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fbd1e-ef52-41ce-9971-eee52188606e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:44.760014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fbd1e-ef52-41ce-9971-eee52188606e None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=3df3fc4e561247979a49d0b052eaee73, user.id=03cc61e3ccb34ecca1eb16cd8199b9e1) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:44.760324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fbd1e-ef52-41ce-9971-eee52188606e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:44.761713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fbd1e-ef52-41ce-9971-eee52188606e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:44.766994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 568/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:53:44 2022] GET /identity/v3/role_assignments?user.id=03cc61e3ccb34ecca1eb16cd8199b9e1&scope.project.id=3df3fc4e561247979a49d0b052eaee73&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:46.740992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18868a8f-5b61-419b-ad7f-4e01a475c766 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:46.741537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18868a8f-5b61-419b-ad7f-4e01a475c766 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:46.741537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18868a8f-5b61-419b-ad7f-4e01a475c766 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:46.742943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 579/1147] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:46.748130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381eb94-ddd5-41d5-8a5c-5cb14a0554a7 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:46.748564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381eb94-ddd5-41d5-8a5c-5cb14a0554a7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:46.748682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381eb94-ddd5-41d5-8a5c-5cb14a0554a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:46.765128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1381eb94-ddd5-41d5-8a5c-5cb14a0554a7 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:46.766350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1381eb94-ddd5-41d5-8a5c-5cb14a0554a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:46.817236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:46.830812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 569/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:46.854462 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18868a8f-5b61-419b-ad7f-4e01a475c766 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:46.953454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18868a8f-5b61-419b-ad7f-4e01a475c766 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:46.982155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:46.987123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46b32d-c9ea-46d8-bd9b-5927a21102fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:46.987896 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46b32d-c9ea-46d8-bd9b-5927a21102fb None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:46.988176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46b32d-c9ea-46d8-bd9b-5927a21102fb None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:46.988446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46b32d-c9ea-46d8-bd9b-5927a21102fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:46.998248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a46b32d-c9ea-46d8-bd9b-5927a21102fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-9a46b32d-c9ea-46d8-bd9b-5927a21102fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:47.005873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:47.006961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:47.008017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 580/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:53:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:47.013130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381eb94-ddd5-41d5-8a5c-5cb14a0554a7 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:47.108278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381eb94-ddd5-41d5-8a5c-5cb14a0554a7 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:47.141078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518ef26-7861-4b2e-a764-afe99e0c7425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:47.141726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518ef26-7861-4b2e-a764-afe99e0c7425 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:47.142009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518ef26-7861-4b2e-a764-afe99e0c7425 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:47.142314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0518ef26-7861-4b2e-a764-afe99e0c7425 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:47.143046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518ef26-7861-4b2e-a764-afe99e0c7425 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:47.143352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518ef26-7861-4b2e-a764-afe99e0c7425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:47.144702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0518ef26-7861-4b2e-a764-afe99e0c7425 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:47.153261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 570/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:53:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:47.158731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46b32d-c9ea-46d8-bd9b-5927a21102fb None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:47.253327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46b32d-c9ea-46d8-bd9b-5927a21102fb None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:47.285542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d2cfd-e0e6-42a2-a680-54db84534469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:47.286414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d2cfd-e0e6-42a2-a680-54db84534469 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:47.286692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d2cfd-e0e6-42a2-a680-54db84534469 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:47.286962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572d2cfd-e0e6-42a2-a680-54db84534469 None admin] PATH_INFO: `/v3/users/03cc61e3ccb34ecca1eb16cd8199b9e1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:47.288117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d2cfd-e0e6-42a2-a680-54db84534469 None admin] RBAC: Authorizing `identity:get_user(user_id=03cc61e3ccb34ecca1eb16cd8199b9e1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:47.289521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572d2cfd-e0e6-42a2-a680-54db84534469 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:47.291174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 581/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:47 2022] GET /identity/v3/users/03cc61e3ccb34ecca1eb16cd8199b9e1 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:47.296613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518ef26-7861-4b2e-a764-afe99e0c7425 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:47.344934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:47.393498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0518ef26-7861-4b2e-a764-afe99e0c7425 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:47.425693 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfad322-2aab-43d2-a495-01679644859a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:47.426563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfad322-2aab-43d2-a495-01679644859a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:47.426849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfad322-2aab-43d2-a495-01679644859a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:47.427126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfad322-2aab-43d2-a495-01679644859a None admin] PATH_INFO: `/v3/projects/3df3fc4e561247979a49d0b052eaee73` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:47.428283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfad322-2aab-43d2-a495-01679644859a None admin] RBAC: Authorizing `identity:get_project(project_id=3df3fc4e561247979a49d0b052eaee73)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:47.429694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfad322-2aab-43d2-a495-01679644859a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:47.431908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 571/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:53:47 2022] GET /identity/v3/projects/3df3fc4e561247979a49d0b052eaee73 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:47.437489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d2cfd-e0e6-42a2-a680-54db84534469 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:47.493996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:47.533579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572d2cfd-e0e6-42a2-a680-54db84534469 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:47.565273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8129d8-9ee7-415b-8f46-3d56d82aea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:47.566208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8129d8-9ee7-415b-8f46-3d56d82aea22 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:47.566490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8129d8-9ee7-415b-8f46-3d56d82aea22 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:47.566762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8129d8-9ee7-415b-8f46-3d56d82aea22 None admin] PATH_INFO: `/v3/projects/3df3fc4e561247979a49d0b052eaee73/users/03cc61e3ccb34ecca1eb16cd8199b9e1/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:47.568764 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8129d8-9ee7-415b-8f46-3d56d82aea22 None admin] RBAC: Authorizing `identity:create_grant(project_id=3df3fc4e561247979a49d0b052eaee73, user_id=03cc61e3ccb34ecca1eb16cd8199b9e1, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:47.570215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8129d8-9ee7-415b-8f46-3d56d82aea22 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:47.583023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 582/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:53:47 2022] PUT /identity/v3/projects/3df3fc4e561247979a49d0b052eaee73/users/03cc61e3ccb34ecca1eb16cd8199b9e1/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:53:49.598245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1000da8-beaa-4a6b-89be-661a802f4f35 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:49.598768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1000da8-beaa-4a6b-89be-661a802f4f35 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:49.598768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1000da8-beaa-4a6b-89be-661a802f4f35 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:49.600186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 572/1154] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:49.605739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc794f-317e-46ef-b10e-4021f3fa3397 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:49.606127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc794f-317e-46ef-b10e-4021f3fa3397 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:49.606332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cc794f-317e-46ef-b10e-4021f3fa3397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:49.623268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b3cc794f-317e-46ef-b10e-4021f3fa3397 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:49.624719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b3cc794f-317e-46ef-b10e-4021f3fa3397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:49.688406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 583/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:49.711212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1000da8-beaa-4a6b-89be-661a802f4f35 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:49.808954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1000da8-beaa-4a6b-89be-661a802f4f35 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:49.824682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:49.841924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf4d7c-530f-4c9b-bc69-ba5cfcf314b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:49.842730 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf4d7c-530f-4c9b-bc69-ba5cfcf314b9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:49.843017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf4d7c-530f-4c9b-bc69-ba5cfcf314b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:49.843295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cf4d7c-530f-4c9b-bc69-ba5cfcf314b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:49.854931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cf4d7c-530f-4c9b-bc69-ba5cfcf314b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-62cf4d7c-530f-4c9b-bc69-ba5cfcf314b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:53:49.862432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:49.863505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:53:49.864509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 573/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:53:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:49.869571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc794f-317e-46ef-b10e-4021f3fa3397 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:49.918061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:49.966007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cc794f-317e-46ef-b10e-4021f3fa3397 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:49.998552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e9b67-1d2a-4c97-be5f-9a1b68fb0994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:49.999474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e9b67-1d2a-4c97-be5f-9a1b68fb0994 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:49.999758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e9b67-1d2a-4c97-be5f-9a1b68fb0994 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:50.000034 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e9b67-1d2a-4c97-be5f-9a1b68fb0994 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:50.000772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e9b67-1d2a-4c97-be5f-9a1b68fb0994 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:50.001078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e9b67-1d2a-4c97-be5f-9a1b68fb0994 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:50.002566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80e9b67-1d2a-4c97-be5f-9a1b68fb0994 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:50.010447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 584/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:53:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:53:50.016024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf4d7c-530f-4c9b-bc69-ba5cfcf314b9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:50.111687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cf4d7c-530f-4c9b-bc69-ba5cfcf314b9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:50.144821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0720d0-00a1-44a6-a35e-1a322266b2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:50.145682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0720d0-00a1-44a6-a35e-1a322266b2e2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:50.145964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0720d0-00a1-44a6-a35e-1a322266b2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:50.146266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0720d0-00a1-44a6-a35e-1a322266b2e2 None admin] PATH_INFO: `/v3/users/03cc61e3ccb34ecca1eb16cd8199b9e1` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:50.147433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0720d0-00a1-44a6-a35e-1a322266b2e2 None admin] RBAC: Authorizing `identity:get_user(user_id=03cc61e3ccb34ecca1eb16cd8199b9e1)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:50.148845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0720d0-00a1-44a6-a35e-1a322266b2e2 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:50.151150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 574/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:53:50 2022] GET /identity/v3/users/03cc61e3ccb34ecca1eb16cd8199b9e1 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:50.156838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e9b67-1d2a-4c97-be5f-9a1b68fb0994 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:50.252793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e9b67-1d2a-4c97-be5f-9a1b68fb0994 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:50.265504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:50.286221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5ba1d-3c1b-44ce-a560-53c3bfb46c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:50.286854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5ba1d-3c1b-44ce-a560-53c3bfb46c74 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:50.287137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5ba1d-3c1b-44ce-a560-53c3bfb46c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:50.287413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5ba1d-3c1b-44ce-a560-53c3bfb46c74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:50.351145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5ba1d-3c1b-44ce-a560-53c3bfb46c74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:50.352600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb5ba1d-3c1b-44ce-a560-53c3bfb46c74 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:50.448137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 585/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:53:50.453979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0720d0-00a1-44a6-a35e-1a322266b2e2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:50.498349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:50.548403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0720d0-00a1-44a6-a35e-1a322266b2e2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:50.580966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8426412-ad06-4e92-aca4-4609147b013a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:50.581801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8426412-ad06-4e92-aca4-4609147b013a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:50.582103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8426412-ad06-4e92-aca4-4609147b013a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:50.582372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8426412-ad06-4e92-aca4-4609147b013a None admin] PATH_INFO: `/v3/projects/3df3fc4e561247979a49d0b052eaee73` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:50.583523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8426412-ad06-4e92-aca4-4609147b013a None admin] RBAC: Authorizing `identity:get_project(project_id=3df3fc4e561247979a49d0b052eaee73)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:50.584930 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8426412-ad06-4e92-aca4-4609147b013a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:50.586666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 575/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:53:50 2022] GET /identity/v3/projects/3df3fc4e561247979a49d0b052eaee73 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:50.592603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5ba1d-3c1b-44ce-a560-53c3bfb46c74 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:50.607243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:50.689388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5ba1d-3c1b-44ce-a560-53c3bfb46c74 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:50.722656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547cdaab-c73a-4307-af36-fdb4b82db7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:50.723242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cdaab-c73a-4307-af36-fdb4b82db7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:50.723526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cdaab-c73a-4307-af36-fdb4b82db7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:50.723803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547cdaab-c73a-4307-af36-fdb4b82db7a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:50.724629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cdaab-c73a-4307-af36-fdb4b82db7a7 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=3df3fc4e561247979a49d0b052eaee73, user.id=03cc61e3ccb34ecca1eb16cd8199b9e1) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:50.724944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cdaab-c73a-4307-af36-fdb4b82db7a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:50.726390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547cdaab-c73a-4307-af36-fdb4b82db7a7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:50.732367 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 586/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:53:50 2022] GET /identity/v3/role_assignments?user.id=03cc61e3ccb34ecca1eb16cd8199b9e1&scope.project.id=3df3fc4e561247979a49d0b052eaee73&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:52.735662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca9cff-0a5f-4c71-9511-e2f1909a586c None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:52.736187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca9cff-0a5f-4c71-9511-e2f1909a586c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:52.736277 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ca9cff-0a5f-4c71-9511-e2f1909a586c None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:52.737623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 576/1162] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:52.742637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56bb62-8e74-4048-b146-3d096d4edb35 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:52.743067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56bb62-8e74-4048-b146-3d096d4edb35 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:52.743202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56bb62-8e74-4048-b146-3d096d4edb35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:52.759734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0c56bb62-8e74-4048-b146-3d096d4edb35 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:52.760927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0c56bb62-8e74-4048-b146-3d096d4edb35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:52.825125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 587/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:52.847784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca9cff-0a5f-4c71-9511-e2f1909a586c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:52.945568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ca9cff-0a5f-4c71-9511-e2f1909a586c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:52.958824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:52.979432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047c1cf-9f0a-4535-97b1-ade8eef7c459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:52.980068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c1cf-9f0a-4535-97b1-ade8eef7c459 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:52.980351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c1cf-9f0a-4535-97b1-ade8eef7c459 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:52.980629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8047c1cf-9f0a-4535-97b1-ade8eef7c459 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:53.044709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047c1cf-9f0a-4535-97b1-ade8eef7c459 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:53.046097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8047c1cf-9f0a-4535-97b1-ade8eef7c459 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:53.139806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 577/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:53:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:53:53.145996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56bb62-8e74-4048-b146-3d096d4edb35 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:53.154025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:53.241628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56bb62-8e74-4048-b146-3d096d4edb35 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:53.274774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4c279f-89b3-428d-9263-27010e019c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:53.275562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c279f-89b3-428d-9263-27010e019c3c None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:53.275846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c279f-89b3-428d-9263-27010e019c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:53.276122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4c279f-89b3-428d-9263-27010e019c3c None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:53.282606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c279f-89b3-428d-9263-27010e019c3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:53.284024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4c279f-89b3-428d-9263-27010e019c3c None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-3a4c279f-89b3-428d-9263-27010e019c3c None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:53:53.298284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:53:53.299434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:53:53.300442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 588/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Jun 26 02:53:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:53:53.305233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047c1cf-9f0a-4535-97b1-ade8eef7c459 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:53.320195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:53.401575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8047c1cf-9f0a-4535-97b1-ade8eef7c459 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:53.433769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1049b76-fc7f-44dd-b459-67549c5cb590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:53.434444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049b76-fc7f-44dd-b459-67549c5cb590 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:53.434731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049b76-fc7f-44dd-b459-67549c5cb590 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:53.435008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1049b76-fc7f-44dd-b459-67549c5cb590 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:53.435723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1049b76-fc7f-44dd-b459-67549c5cb590 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:53:53.436033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1049b76-fc7f-44dd-b459-67549c5cb590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:53.437375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1049b76-fc7f-44dd-b459-67549c5cb590 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:53.444289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 578/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Jun 26 02:53:53 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:55.409143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca616fa0-236f-485a-ae03-1e6c0ec6e770 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:55.409769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca616fa0-236f-485a-ae03-1e6c0ec6e770 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:55.409769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca616fa0-236f-485a-ae03-1e6c0ec6e770 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:55.411130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 589/1167] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:55.416262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbb9a9-afcf-4fb2-abfb-654c7e5fb649 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:55.416645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbb9a9-afcf-4fb2-abfb-654c7e5fb649 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:55.416812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bbb9a9-afcf-4fb2-abfb-654c7e5fb649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:55.433615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-08bbb9a9-afcf-4fb2-abfb-654c7e5fb649 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:55.434792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-08bbb9a9-afcf-4fb2-abfb-654c7e5fb649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:55.497787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 579/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:55.520978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca616fa0-236f-485a-ae03-1e6c0ec6e770 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:55.584943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:55.620237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca616fa0-236f-485a-ae03-1e6c0ec6e770 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:55.653100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e422c7-e204-47d0-95bd-6f1a0da9a9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:55.653741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e422c7-e204-47d0-95bd-6f1a0da9a9f9 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:55.654022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e422c7-e204-47d0-95bd-6f1a0da9a9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:55.654327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e422c7-e204-47d0-95bd-6f1a0da9a9f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:55.655065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e422c7-e204-47d0-95bd-6f1a0da9a9f9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:55.656411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e422c7-e204-47d0-95bd-6f1a0da9a9f9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:55.670205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 590/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Sun Jun 26 02:53:55 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:53:57.686227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20127d5d-04d4-405c-87bc-db082f7c9db9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:57.686775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20127d5d-04d4-405c-87bc-db082f7c9db9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:57.686775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20127d5d-04d4-405c-87bc-db082f7c9db9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:57.688183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 580/1170] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:53:57.693348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73a502-1ddf-4059-84a9-240b962b5e7d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:57.693749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73a502-1ddf-4059-84a9-240b962b5e7d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:57.693913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce73a502-1ddf-4059-84a9-240b962b5e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:57.710763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ce73a502-1ddf-4059-84a9-240b962b5e7d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:53:57.712577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ce73a502-1ddf-4059-84a9-240b962b5e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:53:57.776893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 591/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:53:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:53:57.800108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20127d5d-04d4-405c-87bc-db082f7c9db9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:57.810559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:57.899757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20127d5d-04d4-405c-87bc-db082f7c9db9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:57.931942 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd97fe1-8f2f-420b-93e9-beab7fe36ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:57.932798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd97fe1-8f2f-420b-93e9-beab7fe36ebc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:57.933080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd97fe1-8f2f-420b-93e9-beab7fe36ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:57.933353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd97fe1-8f2f-420b-93e9-beab7fe36ebc None admin] PATH_INFO: `/v3/domains/1b77703d63f0417eac637f988896b6cf` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:57.943327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd97fe1-8f2f-420b-93e9-beab7fe36ebc None admin] RBAC: Authorizing `identity:get_domain(domain_id=1b77703d63f0417eac637f988896b6cf)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:57.944706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd97fe1-8f2f-420b-93e9-beab7fe36ebc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:57.947016 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 581/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Sun Jun 26 02:53:57 2022] GET /identity/v3/domains/1b77703d63f0417eac637f988896b6cf => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:53:57.952822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73a502-1ddf-4059-84a9-240b962b5e7d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:53:58.006844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:53:58.049297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce73a502-1ddf-4059-84a9-240b962b5e7d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:53:58.082435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93876d65-4724-4148-9256-6646a2a20e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:53:58.083086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93876d65-4724-4148-9256-6646a2a20e74 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:53:58.083370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93876d65-4724-4148-9256-6646a2a20e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:53:58.083646 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93876d65-4724-4148-9256-6646a2a20e74 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:53:58.084345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93876d65-4724-4148-9256-6646a2a20e74 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:53:58.085753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93876d65-4724-4148-9256-6646a2a20e74 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:53:58.109013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 592/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:53:57 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:54:00.123150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49659155-d9b1-4d36-8e83-f31e7af44450 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:00.123682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49659155-d9b1-4d36-8e83-f31e7af44450 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:00.123775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49659155-d9b1-4d36-8e83-f31e7af44450 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:00.125392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 582/1174] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:54:00.130792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081ca33-01c9-4cc5-8ecb-91826b054c48 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:00.131179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081ca33-01c9-4cc5-8ecb-91826b054c48 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:00.131360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081ca33-01c9-4cc5-8ecb-91826b054c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:00.147698 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6081ca33-01c9-4cc5-8ecb-91826b054c48 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:54:00.149377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6081ca33-01c9-4cc5-8ecb-91826b054c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:54:00.213648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 593/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:54:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 26 02:54:00.238428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49659155-d9b1-4d36-8e83-f31e7af44450 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:00.303185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:00.335544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49659155-d9b1-4d36-8e83-f31e7af44450 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:00.367858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6812a0-58b9-4735-b3c4-27c478906b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:00.368700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6812a0-58b9-4735-b3c4-27c478906b7e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:00.368986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6812a0-58b9-4735-b3c4-27c478906b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:00.369260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6812a0-58b9-4735-b3c4-27c478906b7e None admin] PATH_INFO: `/v3/domains/1b77703d63f0417eac637f988896b6cf` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:00.370446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6812a0-58b9-4735-b3c4-27c478906b7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=1b77703d63f0417eac637f988896b6cf)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:00.371859 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6812a0-58b9-4735-b3c4-27c478906b7e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:00.373510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 583/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Sun Jun 26 02:54:00 2022] GET /identity/v3/domains/1b77703d63f0417eac637f988896b6cf => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:00.379283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081ca33-01c9-4cc5-8ecb-91826b054c48 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:00.417582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:00.477931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081ca33-01c9-4cc5-8ecb-91826b054c48 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:00.509811 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79b2139-1e1b-4160-b03e-882e6236ef18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:00.510488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b2139-1e1b-4160-b03e-882e6236ef18 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:00.510770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b2139-1e1b-4160-b03e-882e6236ef18 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:00.511042 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79b2139-1e1b-4160-b03e-882e6236ef18 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:00.511748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b2139-1e1b-4160-b03e-882e6236ef18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:00.513156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79b2139-1e1b-4160-b03e-882e6236ef18 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:00.532097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 594/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:54:00 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:54:02.557138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503a369-65c4-41a8-91d6-46bf01dbc76e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:02.557687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503a369-65c4-41a8-91d6-46bf01dbc76e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:02.557687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0503a369-65c4-41a8-91d6-46bf01dbc76e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:02.559272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 584/1178] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:54:02.564517 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527595d5-bfb2-480d-a6d5-f9281162b538 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:02.564936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527595d5-bfb2-480d-a6d5-f9281162b538 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:02.565078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527595d5-bfb2-480d-a6d5-f9281162b538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:02.581978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-527595d5-bfb2-480d-a6d5-f9281162b538 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:54:02.583197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-527595d5-bfb2-480d-a6d5-f9281162b538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:54:02.647425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 595/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:54:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:54:02.670471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503a369-65c4-41a8-91d6-46bf01dbc76e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:02.769846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0503a369-65c4-41a8-91d6-46bf01dbc76e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:02.801884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b379a1-b87f-4982-b747-2210526f29b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:02.802678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b379a1-b87f-4982-b747-2210526f29b8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:02.802966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b379a1-b87f-4982-b747-2210526f29b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:02.803244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b379a1-b87f-4982-b747-2210526f29b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:02.806399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:02.814932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b379a1-b87f-4982-b747-2210526f29b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b1b379a1-b87f-4982-b747-2210526f29b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:54:02.822496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:54:02.823518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:54:02.824624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 585/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:54:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:54:02.829536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527595d5-bfb2-480d-a6d5-f9281162b538 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:02.853728 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:02.926923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527595d5-bfb2-480d-a6d5-f9281162b538 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:02.960549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b743c-5c43-429c-95f5-f0c2b9bb6ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:02.961204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b743c-5c43-429c-95f5-f0c2b9bb6ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:02.961510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b743c-5c43-429c-95f5-f0c2b9bb6ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:02.961787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b743c-5c43-429c-95f5-f0c2b9bb6ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:02.962563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b743c-5c43-429c-95f5-f0c2b9bb6ca3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:54:02.962873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b743c-5c43-429c-95f5-f0c2b9bb6ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:02.964231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962b743c-5c43-429c-95f5-f0c2b9bb6ca3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:02.972051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 596/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:54:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:02.978205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b379a1-b87f-4982-b747-2210526f29b8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:03.073056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b379a1-b87f-4982-b747-2210526f29b8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:03.104776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a93263-96d0-4c2d-94b9-cd6f8b66bedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:03.105617 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93263-96d0-4c2d-94b9-cd6f8b66bedc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:03.105896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93263-96d0-4c2d-94b9-cd6f8b66bedc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:03.106197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a93263-96d0-4c2d-94b9-cd6f8b66bedc None admin] PATH_INFO: `/v3/users/87ec3569e1b5447ca112aa37890bfaed` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:03.119117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a93263-96d0-4c2d-94b9-cd6f8b66bedc None admin] RBAC: Authorizing `identity:get_user(user_id=87ec3569e1b5447ca112aa37890bfaed)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:03.120530 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a93263-96d0-4c2d-94b9-cd6f8b66bedc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:03.122194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 586/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:54:02 2022] GET /identity/v3/users/87ec3569e1b5447ca112aa37890bfaed => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:03.128051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b743c-5c43-429c-95f5-f0c2b9bb6ca3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:03.208530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:03.223693 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b743c-5c43-429c-95f5-f0c2b9bb6ca3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:03.255715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66b899-9180-4ac2-9416-322980eecf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:03.256335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66b899-9180-4ac2-9416-322980eecf31 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:03.256615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66b899-9180-4ac2-9416-322980eecf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:03.256886 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe66b899-9180-4ac2-9416-322980eecf31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:03.320160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66b899-9180-4ac2-9416-322980eecf31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:03.321596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe66b899-9180-4ac2-9416-322980eecf31 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:03.399837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:03.415936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 597/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:54:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:54:03.421705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a93263-96d0-4c2d-94b9-cd6f8b66bedc None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:03.474586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:03.517215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a93263-96d0-4c2d-94b9-cd6f8b66bedc None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:03.550216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46095d2e-86da-4225-8513-b0c37ea9a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:03.551064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46095d2e-86da-4225-8513-b0c37ea9a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:03.551347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46095d2e-86da-4225-8513-b0c37ea9a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:03.551622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46095d2e-86da-4225-8513-b0c37ea9a1b1 None admin] PATH_INFO: `/v3/projects/59ea6e5da25046e28f6fd4b689727049` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:03.552776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46095d2e-86da-4225-8513-b0c37ea9a1b1 None admin] RBAC: Authorizing `identity:get_project(project_id=59ea6e5da25046e28f6fd4b689727049)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:03.554218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46095d2e-86da-4225-8513-b0c37ea9a1b1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:03.555989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 587/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:54:03 2022] GET /identity/v3/projects/59ea6e5da25046e28f6fd4b689727049 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:03.561950 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66b899-9180-4ac2-9416-322980eecf31 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:03.658022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe66b899-9180-4ac2-9416-322980eecf31 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:03.692559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615e15f-e8bd-4972-b758-448bdf2e9366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:03.693200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615e15f-e8bd-4972-b758-448bdf2e9366 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:03.693485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615e15f-e8bd-4972-b758-448bdf2e9366 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:03.693761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615e15f-e8bd-4972-b758-448bdf2e9366 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:03.694618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615e15f-e8bd-4972-b758-448bdf2e9366 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=59ea6e5da25046e28f6fd4b689727049, user.id=87ec3569e1b5447ca112aa37890bfaed) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:54:03.694931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615e15f-e8bd-4972-b758-448bdf2e9366 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:03.696852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615e15f-e8bd-4972-b758-448bdf2e9366 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:03.702181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 598/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:54:03 2022] GET /identity/v3/role_assignments?user.id=87ec3569e1b5447ca112aa37890bfaed&scope.project.id=59ea6e5da25046e28f6fd4b689727049&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:05.658444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792d4c6-e577-4c8e-a652-76a8aa9188a9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:05.658977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792d4c6-e577-4c8e-a652-76a8aa9188a9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:05.659070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792d4c6-e577-4c8e-a652-76a8aa9188a9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:05.660430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 588/1186] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:54:05.665507 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d705f-d7cb-4e9c-947a-655a15aa8d6a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:05.665933 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d705f-d7cb-4e9c-947a-655a15aa8d6a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:05.666096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0d705f-d7cb-4e9c-947a-655a15aa8d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:05.682872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4d0d705f-d7cb-4e9c-947a-655a15aa8d6a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:54:05.684064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4d0d705f-d7cb-4e9c-947a-655a15aa8d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:54:05.728482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:05.749601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 599/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:54:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:54:05.773000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792d4c6-e577-4c8e-a652-76a8aa9188a9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:05.878472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792d4c6-e577-4c8e-a652-76a8aa9188a9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:05.900301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:05.912115 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccbfcc-e8de-4530-a08e-2c81bb79a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:05.912842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccbfcc-e8de-4530-a08e-2c81bb79a9cf None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:05.913129 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccbfcc-e8de-4530-a08e-2c81bb79a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:05.913407 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccbfcc-e8de-4530-a08e-2c81bb79a9cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:05.923799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccbfcc-e8de-4530-a08e-2c81bb79a9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-35ccbfcc-e8de-4530-a08e-2c81bb79a9cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:54:05.932087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:54:05.933222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:54:05.934175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 589/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:54:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:54:05.939457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d705f-d7cb-4e9c-947a-655a15aa8d6a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:06.034334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0d705f-d7cb-4e9c-947a-655a15aa8d6a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:06.067447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5de24f-c9a4-476b-946a-498778fd042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:06.068101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5de24f-c9a4-476b-946a-498778fd042f None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:06.068387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5de24f-c9a4-476b-946a-498778fd042f None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:06.068667 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5de24f-c9a4-476b-946a-498778fd042f None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:06.069406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5de24f-c9a4-476b-946a-498778fd042f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:54:06.069717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5de24f-c9a4-476b-946a-498778fd042f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:06.071137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5de24f-c9a4-476b-946a-498778fd042f None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:06.078952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 600/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:54:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:06.084450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccbfcc-e8de-4530-a08e-2c81bb79a9cf None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:06.180050 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccbfcc-e8de-4530-a08e-2c81bb79a9cf None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:06.210936 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56406b4b-e332-44fa-a740-f5b465890083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:06.212308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56406b4b-e332-44fa-a740-f5b465890083 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:06.212593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56406b4b-e332-44fa-a740-f5b465890083 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:06.212866 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56406b4b-e332-44fa-a740-f5b465890083 None admin] PATH_INFO: `/v3/users/87ec3569e1b5447ca112aa37890bfaed` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:06.214021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56406b4b-e332-44fa-a740-f5b465890083 None admin] RBAC: Authorizing `identity:get_user(user_id=87ec3569e1b5447ca112aa37890bfaed)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:06.215451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56406b4b-e332-44fa-a740-f5b465890083 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:06.217086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 590/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:54:06 2022] GET /identity/v3/users/87ec3569e1b5447ca112aa37890bfaed => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:06.222819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5de24f-c9a4-476b-946a-498778fd042f None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:06.265603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:06.320452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5de24f-c9a4-476b-946a-498778fd042f None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:06.352799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b965fd63-f012-4065-b0d6-d9ba96ad6934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:06.353629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965fd63-f012-4065-b0d6-d9ba96ad6934 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:06.353907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965fd63-f012-4065-b0d6-d9ba96ad6934 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:06.354202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b965fd63-f012-4065-b0d6-d9ba96ad6934 None admin] PATH_INFO: `/v3/projects/59ea6e5da25046e28f6fd4b689727049` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:06.355369 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965fd63-f012-4065-b0d6-d9ba96ad6934 None admin] RBAC: Authorizing `identity:get_project(project_id=59ea6e5da25046e28f6fd4b689727049)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:06.356775 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b965fd63-f012-4065-b0d6-d9ba96ad6934 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:06.358488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 601/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:54:06 2022] GET /identity/v3/projects/59ea6e5da25046e28f6fd4b689727049 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:06.364314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56406b4b-e332-44fa-a740-f5b465890083 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:06.412199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:06.474336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56406b4b-e332-44fa-a740-f5b465890083 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:06.508963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be27cd7f-d77c-4ccc-87d1-31eb855a0afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:06.509840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27cd7f-d77c-4ccc-87d1-31eb855a0afd None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:06.510146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27cd7f-d77c-4ccc-87d1-31eb855a0afd None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:06.510419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be27cd7f-d77c-4ccc-87d1-31eb855a0afd None admin] PATH_INFO: `/v3/projects/59ea6e5da25046e28f6fd4b689727049/users/87ec3569e1b5447ca112aa37890bfaed/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:06.512354 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27cd7f-d77c-4ccc-87d1-31eb855a0afd None admin] RBAC: Authorizing `identity:create_grant(project_id=59ea6e5da25046e28f6fd4b689727049, user_id=87ec3569e1b5447ca112aa37890bfaed, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:06.513777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be27cd7f-d77c-4ccc-87d1-31eb855a0afd None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:06.526375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 591/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:54:06 2022] PUT /identity/v3/projects/59ea6e5da25046e28f6fd4b689727049/users/87ec3569e1b5447ca112aa37890bfaed/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:54:08.488499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81322d6f-5fac-4f33-bc3e-9acbc379a83d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:08.489086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81322d6f-5fac-4f33-bc3e-9acbc379a83d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:08.489086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81322d6f-5fac-4f33-bc3e-9acbc379a83d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:08.490461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 602/1193] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:54:08.495673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d142f0-8888-4e13-ac1a-ae3b6684daea None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:08.496066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d142f0-8888-4e13-ac1a-ae3b6684daea None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:08.496236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d142f0-8888-4e13-ac1a-ae3b6684daea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:08.512684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c1d142f0-8888-4e13-ac1a-ae3b6684daea None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:54:08.513858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c1d142f0-8888-4e13-ac1a-ae3b6684daea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:54:08.577182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 592/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:54:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:54:08.600591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81322d6f-5fac-4f33-bc3e-9acbc379a83d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:08.697366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81322d6f-5fac-4f33-bc3e-9acbc379a83d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:08.707029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:08.730575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d29002-0e96-4dd3-82ce-7a0ddbc3ec3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:08.731339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d29002-0e96-4dd3-82ce-7a0ddbc3ec3d None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:08.731618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d29002-0e96-4dd3-82ce-7a0ddbc3ec3d None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:08.731889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d29002-0e96-4dd3-82ce-7a0ddbc3ec3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:08.742154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d29002-0e96-4dd3-82ce-7a0ddbc3ec3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-b3d29002-0e96-4dd3-82ce-7a0ddbc3ec3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:54:08.749739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:54:08.750926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:54:08.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 603/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:54:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:54:08.756826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d142f0-8888-4e13-ac1a-ae3b6684daea None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:08.797100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:08.853322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d142f0-8888-4e13-ac1a-ae3b6684daea None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:08.886383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f101b-5dca-4c39-b324-8afba92cd293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:08.887043 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f101b-5dca-4c39-b324-8afba92cd293 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:08.887323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f101b-5dca-4c39-b324-8afba92cd293 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:08.887597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5f101b-5dca-4c39-b324-8afba92cd293 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:08.888334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f101b-5dca-4c39-b324-8afba92cd293 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:54:08.888639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f101b-5dca-4c39-b324-8afba92cd293 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:08.889987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5f101b-5dca-4c39-b324-8afba92cd293 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:08.897789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 593/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:54:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:08.904143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d29002-0e96-4dd3-82ce-7a0ddbc3ec3d None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:08.997537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d29002-0e96-4dd3-82ce-7a0ddbc3ec3d None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:09.030257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae06458-383d-4e63-93f5-c68bc777a1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:09.031099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae06458-383d-4e63-93f5-c68bc777a1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:09.031379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae06458-383d-4e63-93f5-c68bc777a1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:09.031650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae06458-383d-4e63-93f5-c68bc777a1c0 None admin] PATH_INFO: `/v3/users/87ec3569e1b5447ca112aa37890bfaed` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:09.032799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae06458-383d-4e63-93f5-c68bc777a1c0 None admin] RBAC: Authorizing `identity:get_user(user_id=87ec3569e1b5447ca112aa37890bfaed)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:09.034235 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae06458-383d-4e63-93f5-c68bc777a1c0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:09.035835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 604/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Jun 26 02:54:08 2022] GET /identity/v3/users/87ec3569e1b5447ca112aa37890bfaed => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:54:09.041292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f101b-5dca-4c39-b324-8afba92cd293 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:09.139295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5f101b-5dca-4c39-b324-8afba92cd293 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:09.140748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:09.172023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c4095-e841-448c-80f7-de040ddf14f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:09.172652 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4095-e841-448c-80f7-de040ddf14f8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:09.172932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4095-e841-448c-80f7-de040ddf14f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:09.173202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c4095-e841-448c-80f7-de040ddf14f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:09.236633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c4095-e841-448c-80f7-de040ddf14f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:09.238080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c4095-e841-448c-80f7-de040ddf14f8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:09.330977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 594/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:54:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:54:09.336661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae06458-383d-4e63-93f5-c68bc777a1c0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:09.374982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:09.432108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae06458-383d-4e63-93f5-c68bc777a1c0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:09.464277 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034bd095-e929-407e-bc47-ac47660137f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:09.465107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bd095-e929-407e-bc47-ac47660137f3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:09.465386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bd095-e929-407e-bc47-ac47660137f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:09.465656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034bd095-e929-407e-bc47-ac47660137f3 None admin] PATH_INFO: `/v3/projects/59ea6e5da25046e28f6fd4b689727049` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:09.466934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034bd095-e929-407e-bc47-ac47660137f3 None admin] RBAC: Authorizing `identity:get_project(project_id=59ea6e5da25046e28f6fd4b689727049)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:09.468331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034bd095-e929-407e-bc47-ac47660137f3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:09.469998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 605/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Jun 26 02:54:09 2022] GET /identity/v3/projects/59ea6e5da25046e28f6fd4b689727049 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:09.475699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c4095-e841-448c-80f7-de040ddf14f8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:54:09.477792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:09.570483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c4095-e841-448c-80f7-de040ddf14f8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:54:09.601789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf4c65a-e253-4ee0-b9ee-3e44a2a652eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:54:09.602891 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4c65a-e253-4ee0-b9ee-3e44a2a652eb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:54:09.603170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4c65a-e253-4ee0-b9ee-3e44a2a652eb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:54:09.603438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf4c65a-e253-4ee0-b9ee-3e44a2a652eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:54:09.604255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4c65a-e253-4ee0-b9ee-3e44a2a652eb None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=59ea6e5da25046e28f6fd4b689727049, user.id=87ec3569e1b5447ca112aa37890bfaed) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:54:09.604564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4c65a-e253-4ee0-b9ee-3e44a2a652eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:54:09.605968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf4c65a-e253-4ee0-b9ee-3e44a2a652eb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:54:09.611647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 595/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:54:09 2022] GET /identity/v3/role_assignments?user.id=87ec3569e1b5447ca112aa37890bfaed&scope.project.id=59ea6e5da25046e28f6fd4b689727049&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:54:19.460308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=47 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:54:19.609012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=67 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:04.425451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8403a47-b464-493e-a0e6-f7a8cee1ac31 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:04.426102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8403a47-b464-493e-a0e6-f7a8cee1ac31 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:04.426102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8403a47-b464-493e-a0e6-f7a8cee1ac31 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:04.427621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 606/1201] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:04.433922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c990541-0816-4ee8-a3a0-5dacb6ab0e08 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:04.434457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c990541-0816-4ee8-a3a0-5dacb6ab0e08 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:04.434548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c990541-0816-4ee8-a3a0-5dacb6ab0e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:04.452140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1c990541-0816-4ee8-a3a0-5dacb6ab0e08 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:04.453316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1c990541-0816-4ee8-a3a0-5dacb6ab0e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:04.537547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 596/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:04.564570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8403a47-b464-493e-a0e6-f7a8cee1ac31 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:04.693860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8403a47-b464-493e-a0e6-f7a8cee1ac31 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:04.734710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241d3eb-bd51-43d9-a0e2-2026f740fb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:04.735514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241d3eb-bd51-43d9-a0e2-2026f740fb10 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:04.735795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241d3eb-bd51-43d9-a0e2-2026f740fb10 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:04.736066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0241d3eb-bd51-43d9-a0e2-2026f740fb10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:04.748124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241d3eb-bd51-43d9-a0e2-2026f740fb10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:04.750129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0241d3eb-bd51-43d9-a0e2-2026f740fb10 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0241d3eb-bd51-43d9-a0e2-2026f740fb10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:04.771785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:04.773039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:04.774980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:04.774980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:04.774980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:04.774980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:04.774980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 607/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:55:04.781118 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c990541-0816-4ee8-a3a0-5dacb6ab0e08 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:04.908704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c990541-0816-4ee8-a3a0-5dacb6ab0e08 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:04.943811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:04.952112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fb607-e0cb-498e-b80d-f055258a532c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:04.952740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fb607-e0cb-498e-b80d-f055258a532c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:04.953067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fb607-e0cb-498e-b80d-f055258a532c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:04.953366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fb607-e0cb-498e-b80d-f055258a532c None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:04.954215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fb607-e0cb-498e-b80d-f055258a532c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:04.954541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fb607-e0cb-498e-b80d-f055258a532c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:04.956009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fb607-e0cb-498e-b80d-f055258a532c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:04.983190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 597/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:04.987002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241d3eb-bd51-43d9-a0e2-2026f740fb10 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:05.065542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:05.134184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0241d3eb-bd51-43d9-a0e2-2026f740fb10 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:05.186505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d59844-e6e9-4874-a55f-796e420986a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:05.187952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59844-e6e9-4874-a55f-796e420986a9 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:05.187952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59844-e6e9-4874-a55f-796e420986a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:05.187952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d59844-e6e9-4874-a55f-796e420986a9 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:05.188892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d59844-e6e9-4874-a55f-796e420986a9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:05.190456 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d59844-e6e9-4874-a55f-796e420986a9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:05.212960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 608/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Jun 26 02:55:04 2022] POST /identity/v3/users => generated 251 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:55:07.392594 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ee44-53b5-4709-8cd7-27c1d3a8b956 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:07.393151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ee44-53b5-4709-8cd7-27c1d3a8b956 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:07.393151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d00ee44-53b5-4709-8cd7-27c1d3a8b956 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:07.394802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 598/1206] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:07.401122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f1dbd-36e9-400b-a801-7d0a309db1fc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:07.403482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f1dbd-36e9-400b-a801-7d0a309db1fc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:07.403482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f1dbd-36e9-400b-a801-7d0a309db1fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:07.422300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d52f1dbd-36e9-400b-a801-7d0a309db1fc None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:07.423542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d52f1dbd-36e9-400b-a801-7d0a309db1fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:07.514108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 609/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:07.540808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ee44-53b5-4709-8cd7-27c1d3a8b956 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:07.667705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d00ee44-53b5-4709-8cd7-27c1d3a8b956 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:07.705472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba67a9-9916-4e86-bd61-75ac2f0f7559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:07.706286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba67a9-9916-4e86-bd61-75ac2f0f7559 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:07.706575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba67a9-9916-4e86-bd61-75ac2f0f7559 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:07.706874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ba67a9-9916-4e86-bd61-75ac2f0f7559 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:07.718364 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ba67a9-9916-4e86-bd61-75ac2f0f7559 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b1ba67a9-9916-4e86-bd61-75ac2f0f7559 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:55:07.726886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:07.728064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:07.728963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 599/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:55:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:07.737357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f1dbd-36e9-400b-a801-7d0a309db1fc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:07.786531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:07.868007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f1dbd-36e9-400b-a801-7d0a309db1fc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:07.911495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320cfb6-01bf-4b87-99f8-4f2900d32ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:07.912174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320cfb6-01bf-4b87-99f8-4f2900d32ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:07.912459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320cfb6-01bf-4b87-99f8-4f2900d32ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:07.912764 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320cfb6-01bf-4b87-99f8-4f2900d32ed5 None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:07.913690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320cfb6-01bf-4b87-99f8-4f2900d32ed5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:07.914083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320cfb6-01bf-4b87-99f8-4f2900d32ed5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:07.915754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320cfb6-01bf-4b87-99f8-4f2900d32ed5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:07.926591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 610/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:55:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:55:07.932328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba67a9-9916-4e86-bd61-75ac2f0f7559 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:07.988819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:08.049649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ba67a9-9916-4e86-bd61-75ac2f0f7559 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:08.092011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e0726-933f-418d-9fa9-78c63738a400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:08.092820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e0726-933f-418d-9fa9-78c63738a400 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:08.093109 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e0726-933f-418d-9fa9-78c63738a400 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:08.093390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13e0726-933f-418d-9fa9-78c63738a400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:08.105931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e0726-933f-418d-9fa9-78c63738a400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:08.107501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13e0726-933f-418d-9fa9-78c63738a400 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b13e0726-933f-418d-9fa9-78c63738a400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:08.127552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:08.128966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:08.130675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:08.130675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:08.130675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:08.130675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:08.130675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 600/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:08.135186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320cfb6-01bf-4b87-99f8-4f2900d32ed5 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:08.259966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:08.263431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320cfb6-01bf-4b87-99f8-4f2900d32ed5 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:08.302955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730359ba-a571-4b83-9ab4-d9f2e825f6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:08.302955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730359ba-a571-4b83-9ab4-d9f2e825f6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:08.302955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730359ba-a571-4b83-9ab4-d9f2e825f6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:08.302955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730359ba-a571-4b83-9ab4-d9f2e825f6e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:08.304458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730359ba-a571-4b83-9ab4-d9f2e825f6e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:08.304842 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730359ba-a571-4b83-9ab4-d9f2e825f6e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:08.306308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730359ba-a571-4b83-9ab4-d9f2e825f6e2 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:08.319539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 611/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:55:08.325138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e0726-933f-418d-9fa9-78c63738a400 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:08.445341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13e0726-933f-418d-9fa9-78c63738a400 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:08.447076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:08.487091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334acdd8-2c75-4020-b8c4-7471549deca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:08.488068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334acdd8-2c75-4020-b8c4-7471549deca1 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:08.488403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334acdd8-2c75-4020-b8c4-7471549deca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:08.488757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334acdd8-2c75-4020-b8c4-7471549deca1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:08.498338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334acdd8-2c75-4020-b8c4-7471549deca1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:08.500246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334acdd8-2c75-4020-b8c4-7471549deca1 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-334acdd8-2c75-4020-b8c4-7471549deca1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:55:08.515869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:08.521667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:08.523439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 601/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:08.524168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730359ba-a571-4b83-9ab4-d9f2e825f6e2 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:08.649826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730359ba-a571-4b83-9ab4-d9f2e825f6e2 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:08.695423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc28e19-9503-4b01-b35b-48b9f23d4881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:08.696136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc28e19-9503-4b01-b35b-48b9f23d4881 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:08.696420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc28e19-9503-4b01-b35b-48b9f23d4881 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:08.696695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc28e19-9503-4b01-b35b-48b9f23d4881 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:08.697534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc28e19-9503-4b01-b35b-48b9f23d4881 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:08.697845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc28e19-9503-4b01-b35b-48b9f23d4881 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:08.699353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc28e19-9503-4b01-b35b-48b9f23d4881 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:08.720262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 612/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Sun Jun 26 02:55:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:55:08.726478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334acdd8-2c75-4020-b8c4-7471549deca1 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:08.848781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334acdd8-2c75-4020-b8c4-7471549deca1 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:08.895973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40156ade-52f9-4397-a875-4dc446a54978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:08.896602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40156ade-52f9-4397-a875-4dc446a54978 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:08.896889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40156ade-52f9-4397-a875-4dc446a54978 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:08.897165 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40156ade-52f9-4397-a875-4dc446a54978 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:08.935216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:08.982303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40156ade-52f9-4397-a875-4dc446a54978 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:08.984141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40156ade-52f9-4397-a875-4dc446a54978 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:09.106388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 602/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:55:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:55:09.110885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc28e19-9503-4b01-b35b-48b9f23d4881 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:09.133333 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:09.232497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc28e19-9503-4b01-b35b-48b9f23d4881 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:09.269122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9faac2-33e2-42b2-bf78-d254aaf2f46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:09.269950 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9faac2-33e2-42b2-bf78-d254aaf2f46a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:09.270241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9faac2-33e2-42b2-bf78-d254aaf2f46a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:09.270654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9faac2-33e2-42b2-bf78-d254aaf2f46a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:09.281082 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9faac2-33e2-42b2-bf78-d254aaf2f46a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:09.282586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9faac2-33e2-42b2-bf78-d254aaf2f46a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bd9faac2-33e2-42b2-bf78-d254aaf2f46a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:09.300658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:09.301754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:09.302972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:09.302972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:09.302972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:09.302972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:09.302972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 613/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:55:09.308788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40156ade-52f9-4397-a875-4dc446a54978 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:09.378308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:09.423079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40156ade-52f9-4397-a875-4dc446a54978 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:09.465589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cbf06-aa58-4073-9752-48fae12a2c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:09.466298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cbf06-aa58-4073-9752-48fae12a2c37 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:09.467633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cbf06-aa58-4073-9752-48fae12a2c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:09.467633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51cbf06-aa58-4073-9752-48fae12a2c37 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:09.467833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cbf06-aa58-4073-9752-48fae12a2c37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:09.468007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cbf06-aa58-4073-9752-48fae12a2c37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:09.469365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51cbf06-aa58-4073-9752-48fae12a2c37 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:09.482831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 603/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:09.489275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9faac2-33e2-42b2-bf78-d254aaf2f46a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:09.565716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:09.604481 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9faac2-33e2-42b2-bf78-d254aaf2f46a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:09.645590 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19b90-222b-42a4-a957-e6ea9601f9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:09.646585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19b90-222b-42a4-a957-e6ea9601f9d6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:09.646986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19b90-222b-42a4-a957-e6ea9601f9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:09.647353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed19b90-222b-42a4-a957-e6ea9601f9d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:09.655521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed19b90-222b-42a4-a957-e6ea9601f9d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:09.657143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed19b90-222b-42a4-a957-e6ea9601f9d6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-bed19b90-222b-42a4-a957-e6ea9601f9d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:55:09.673046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:09.674362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:09.675464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:55:09.675464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:55:09.675464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:55:09.675464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:55:09.675464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:55:09.675464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:55:09.675464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:09.675464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:09.677174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 614/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:55:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:09.683276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cbf06-aa58-4073-9752-48fae12a2c37 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:09.809512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51cbf06-aa58-4073-9752-48fae12a2c37 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:09.839931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:09.861246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69cd519-ed56-4c62-8731-0a8a2cfa626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:09.861919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cd519-ed56-4c62-8731-0a8a2cfa626c None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:09.862261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cd519-ed56-4c62-8731-0a8a2cfa626c None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:09.862512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69cd519-ed56-4c62-8731-0a8a2cfa626c None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:09.863308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69cd519-ed56-4c62-8731-0a8a2cfa626c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:09.863617 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69cd519-ed56-4c62-8731-0a8a2cfa626c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:09.865050 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69cd519-ed56-4c62-8731-0a8a2cfa626c None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:09.879491 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 604/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:55:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:09.885859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19b90-222b-42a4-a957-e6ea9601f9d6 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:10.005024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed19b90-222b-42a4-a957-e6ea9601f9d6 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:10.037776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:10.045386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f885e1ab-0db6-46b5-ad5a-42f44d2a8096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:10.046259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885e1ab-0db6-46b5-ad5a-42f44d2a8096 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:10.046580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885e1ab-0db6-46b5-ad5a-42f44d2a8096 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:10.046869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f885e1ab-0db6-46b5-ad5a-42f44d2a8096 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:10.047734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885e1ab-0db6-46b5-ad5a-42f44d2a8096 None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=19ce6f259c22441486bfc04811a271b8) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:10.048051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885e1ab-0db6-46b5-ad5a-42f44d2a8096 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:10.049664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f885e1ab-0db6-46b5-ad5a-42f44d2a8096 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:10.056175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 615/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:55:09 2022] GET /identity/v3/role_assignments?user.id=19ce6f259c22441486bfc04811a271b8&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:55:12.218191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc5a56-ac4f-41fe-9a9a-c2c03977b09c None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:12.218716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc5a56-ac4f-41fe-9a9a-c2c03977b09c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:12.218841 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cc5a56-ac4f-41fe-9a9a-c2c03977b09c None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:12.220140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 605/1220] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:12.225418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b56bc-95dd-4a3c-b5f8-4a4d8c76dbe8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:12.225815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b56bc-95dd-4a3c-b5f8-4a4d8c76dbe8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:12.226024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56b56bc-95dd-4a3c-b5f8-4a4d8c76dbe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:12.243966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a56b56bc-95dd-4a3c-b5f8-4a4d8c76dbe8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:12.245143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a56b56bc-95dd-4a3c-b5f8-4a4d8c76dbe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:12.313189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 616/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:12.338639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc5a56-ac4f-41fe-9a9a-c2c03977b09c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:12.443191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cc5a56-ac4f-41fe-9a9a-c2c03977b09c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:12.477015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd125a-9bdb-4e54-bd78-2cbf338b7482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:12.477731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd125a-9bdb-4e54-bd78-2cbf338b7482 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:12.478019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd125a-9bdb-4e54-bd78-2cbf338b7482 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:12.478318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fd125a-9bdb-4e54-bd78-2cbf338b7482 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:12.488538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fd125a-9bdb-4e54-bd78-2cbf338b7482 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b7fd125a-9bdb-4e54-bd78-2cbf338b7482 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:55:12.496782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:12.498612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:12.500539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 606/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:55:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:12.509174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b56bc-95dd-4a3c-b5f8-4a4d8c76dbe8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:12.625262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56b56bc-95dd-4a3c-b5f8-4a4d8c76dbe8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:12.653617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:12.667111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb51a32-3aca-4798-ab80-338f795e37ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:12.667778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb51a32-3aca-4798-ab80-338f795e37ca None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:12.668062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb51a32-3aca-4798-ab80-338f795e37ca None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:12.668337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb51a32-3aca-4798-ab80-338f795e37ca None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:12.669073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb51a32-3aca-4798-ab80-338f795e37ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:12.669378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb51a32-3aca-4798-ab80-338f795e37ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:12.670768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb51a32-3aca-4798-ab80-338f795e37ca None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:12.679279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 617/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:55:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:12.684974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd125a-9bdb-4e54-bd78-2cbf338b7482 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:12.732466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:12.792242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fd125a-9bdb-4e54-bd78-2cbf338b7482 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:12.825111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312da842-3b6f-4772-9be2-78584dca5a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:12.825893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da842-3b6f-4772-9be2-78584dca5a57 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:12.826209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da842-3b6f-4772-9be2-78584dca5a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:12.826485 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312da842-3b6f-4772-9be2-78584dca5a57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:12.837040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312da842-3b6f-4772-9be2-78584dca5a57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:12.838503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312da842-3b6f-4772-9be2-78584dca5a57 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-312da842-3b6f-4772-9be2-78584dca5a57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:12.857228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:12.858361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:12.859262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:12.859262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:12.859262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:12.859262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:12.859262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 607/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:12.865219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb51a32-3aca-4798-ab80-338f795e37ca None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:12.977725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb51a32-3aca-4798-ab80-338f795e37ca None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:13.014210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa8de6-4b25-4174-9c5c-05af2dccf7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:13.014830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa8de6-4b25-4174-9c5c-05af2dccf7df None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:13.015172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa8de6-4b25-4174-9c5c-05af2dccf7df None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:13.015480 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa8de6-4b25-4174-9c5c-05af2dccf7df None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:13.016224 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa8de6-4b25-4174-9c5c-05af2dccf7df None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:13.016527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa8de6-4b25-4174-9c5c-05af2dccf7df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:13.017882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fa8de6-4b25-4174-9c5c-05af2dccf7df None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:13.030151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 618/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:55:13.036544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312da842-3b6f-4772-9be2-78584dca5a57 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:13.144460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:13.153185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312da842-3b6f-4772-9be2-78584dca5a57 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:13.191492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561999f-3679-4468-abe9-46f6c126dbf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:13.192436 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561999f-3679-4468-abe9-46f6c126dbf5 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:13.192743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561999f-3679-4468-abe9-46f6c126dbf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:13.193074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6561999f-3679-4468-abe9-46f6c126dbf5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:13.202031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561999f-3679-4468-abe9-46f6c126dbf5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:13.203813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6561999f-3679-4468-abe9-46f6c126dbf5 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-6561999f-3679-4468-abe9-46f6c126dbf5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:55:13.220565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:13.222179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:13.223347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 608/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:13.228450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa8de6-4b25-4174-9c5c-05af2dccf7df None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:13.281024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:13.341130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fa8de6-4b25-4174-9c5c-05af2dccf7df None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:13.378427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2aef7-25f8-4541-ba71-7039572acdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:13.379158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2aef7-25f8-4541-ba71-7039572acdde None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:13.379457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2aef7-25f8-4541-ba71-7039572acdde None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:13.379778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2aef7-25f8-4541-ba71-7039572acdde None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:13.380640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2aef7-25f8-4541-ba71-7039572acdde None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:13.380984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2aef7-25f8-4541-ba71-7039572acdde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:13.383174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2aef7-25f8-4541-ba71-7039572acdde None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:13.402637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 619/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Sun Jun 26 02:55:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:13.409038 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561999f-3679-4468-abe9-46f6c126dbf5 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:13.520115 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6561999f-3679-4468-abe9-46f6c126dbf5 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:13.556553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a695c-21fb-4934-bd40-58df60e71c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:13.557430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a695c-21fb-4934-bd40-58df60e71c76 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:13.557725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a695c-21fb-4934-bd40-58df60e71c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:13.558023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1a695c-21fb-4934-bd40-58df60e71c76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:13.570094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a695c-21fb-4934-bd40-58df60e71c76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:13.571699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1a695c-21fb-4934-bd40-58df60e71c76 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-3f1a695c-21fb-4934-bd40-58df60e71c76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:13.589127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:13.590258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:13.591256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:13.591256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:13.591256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:13.591256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:13.591256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 609/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:13.596064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2aef7-25f8-4541-ba71-7039572acdde None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:13.709271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2aef7-25f8-4541-ba71-7039572acdde None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:13.711742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:13.745989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bcad-bf1e-4d86-91d6-cd630187421a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:13.746673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bcad-bf1e-4d86-91d6-cd630187421a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:13.746958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bcad-bf1e-4d86-91d6-cd630187421a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:13.747238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32bcad-bf1e-4d86-91d6-cd630187421a None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:13.747955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bcad-bf1e-4d86-91d6-cd630187421a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:13.748263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bcad-bf1e-4d86-91d6-cd630187421a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:13.749882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32bcad-bf1e-4d86-91d6-cd630187421a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:13.766198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 620/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:55:13.768672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a695c-21fb-4934-bd40-58df60e71c76 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:13.778011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:13.877194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1a695c-21fb-4934-bd40-58df60e71c76 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:13.911259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d179c-d065-4bdc-b51b-6b052347cbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:13.912044 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d179c-d065-4bdc-b51b-6b052347cbf8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:13.912327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d179c-d065-4bdc-b51b-6b052347cbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:13.912596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3d179c-d065-4bdc-b51b-6b052347cbf8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:13.919124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d179c-d065-4bdc-b51b-6b052347cbf8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:13.920557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3d179c-d065-4bdc-b51b-6b052347cbf8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-bd3d179c-d065-4bdc-b51b-6b052347cbf8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:55:13.935966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:13.937007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:13.938001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:55:13.938001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:55:13.938001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:55:13.938001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:55:13.938001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:55:13.938001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:55:13.938001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:13.938001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:13.938001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 610/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:55:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:55:13.944860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bcad-bf1e-4d86-91d6-cd630187421a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:14.057886 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32bcad-bf1e-4d86-91d6-cd630187421a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:14.095184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a4fccb-d406-4ab0-a066-c1035943d7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:14.095900 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4fccb-d406-4ab0-a066-c1035943d7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:14.096184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4fccb-d406-4ab0-a066-c1035943d7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:14.096513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a4fccb-d406-4ab0-a066-c1035943d7d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:14.097361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4fccb-d406-4ab0-a066-c1035943d7d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:14.097676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4fccb-d406-4ab0-a066-c1035943d7d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:14.099149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a4fccb-d406-4ab0-a066-c1035943d7d9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:14.112279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 621/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:55:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:14.120395 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d179c-d065-4bdc-b51b-6b052347cbf8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:14.194346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:14.232304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3d179c-d065-4bdc-b51b-6b052347cbf8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:14.269783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07dcb11-f071-439d-b97d-2fb5fed97293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:14.270694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcb11-f071-439d-b97d-2fb5fed97293 None admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:14.270973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcb11-f071-439d-b97d-2fb5fed97293 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:14.271243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07dcb11-f071-439d-b97d-2fb5fed97293 None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/19ce6f259c22441486bfc04811a271b8/roles/10b1c5eed2754da5adfd5217256adba8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:14.289172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07dcb11-f071-439d-b97d-2fb5fed97293 None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=19ce6f259c22441486bfc04811a271b8, role_id=10b1c5eed2754da5adfd5217256adba8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:14.290769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07dcb11-f071-439d-b97d-2fb5fed97293 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:14.304243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 611/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:55:14 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/19ce6f259c22441486bfc04811a271b8/roles/10b1c5eed2754da5adfd5217256adba8 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 02:55:16.471366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdf3b9-5d52-4ea9-adf5-8567f0d141e8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:16.471970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdf3b9-5d52-4ea9-adf5-8567f0d141e8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:16.472132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccdf3b9-5d52-4ea9-adf5-8567f0d141e8 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:16.474103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 622/1233] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:16.479704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3439efb-a33b-4856-ad41-6ed05f54f2b8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:16.480131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3439efb-a33b-4856-ad41-6ed05f54f2b8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:16.480253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3439efb-a33b-4856-ad41-6ed05f54f2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:16.499061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f3439efb-a33b-4856-ad41-6ed05f54f2b8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:16.500715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f3439efb-a33b-4856-ad41-6ed05f54f2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:16.581101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 612/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:16.606009 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdf3b9-5d52-4ea9-adf5-8567f0d141e8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:16.633643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:16.716396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccdf3b9-5d52-4ea9-adf5-8567f0d141e8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:16.752608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925e6d9-254e-40bc-8a52-57759d41cb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:16.753365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925e6d9-254e-40bc-8a52-57759d41cb8a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:16.753650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925e6d9-254e-40bc-8a52-57759d41cb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:16.753965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0925e6d9-254e-40bc-8a52-57759d41cb8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:16.764292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0925e6d9-254e-40bc-8a52-57759d41cb8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0925e6d9-254e-40bc-8a52-57759d41cb8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:55:16.771974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:16.773614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:16.779819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 623/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Jun 26 02:55:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:16.780670 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3439efb-a33b-4856-ad41-6ed05f54f2b8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:16.830571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:16.888040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3439efb-a33b-4856-ad41-6ed05f54f2b8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:16.923874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30605d-aa13-4b57-8090-a879989d6217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:16.924554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30605d-aa13-4b57-8090-a879989d6217 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:16.924841 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30605d-aa13-4b57-8090-a879989d6217 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:16.925126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc30605d-aa13-4b57-8090-a879989d6217 None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:16.925870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30605d-aa13-4b57-8090-a879989d6217 None admin] RBAC: Adding query filter params (name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:16.926213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30605d-aa13-4b57-8090-a879989d6217 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:16.927598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc30605d-aa13-4b57-8090-a879989d6217 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:16.936343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 613/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Jun 26 02:55:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:16.942677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925e6d9-254e-40bc-8a52-57759d41cb8a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:17.032343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:17.050561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0925e6d9-254e-40bc-8a52-57759d41cb8a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:17.090746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927a6dd-9a71-491d-9e13-684d786c6250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:17.091547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927a6dd-9a71-491d-9e13-684d786c6250 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:17.091844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927a6dd-9a71-491d-9e13-684d786c6250 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:17.092106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4927a6dd-9a71-491d-9e13-684d786c6250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:17.103857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927a6dd-9a71-491d-9e13-684d786c6250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:17.105306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4927a6dd-9a71-491d-9e13-684d786c6250 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-4927a6dd-9a71-491d-9e13-684d786c6250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:17.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:17.123615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:17.124947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:17.124947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:17.124947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:17.124947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:17.124947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 624/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:17.129925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30605d-aa13-4b57-8090-a879989d6217 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:17.233330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc30605d-aa13-4b57-8090-a879989d6217 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:17.244221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:17.273034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc9da7-b5a6-4ea5-bc76-819e4a80ec59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:17.274028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc9da7-b5a6-4ea5-bc76-819e4a80ec59 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:17.274527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc9da7-b5a6-4ea5-bc76-819e4a80ec59 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:17.275047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc9da7-b5a6-4ea5-bc76-819e4a80ec59 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:17.276118 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc9da7-b5a6-4ea5-bc76-819e4a80ec59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:17.276553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc9da7-b5a6-4ea5-bc76-819e4a80ec59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:17.278506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc9da7-b5a6-4ea5-bc76-819e4a80ec59 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:17.292987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 614/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:17.299351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927a6dd-9a71-491d-9e13-684d786c6250 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:17.415455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4927a6dd-9a71-491d-9e13-684d786c6250 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:17.461575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c9a65-c455-4eaf-a0f0-f2683b10fa89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:17.462796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c9a65-c455-4eaf-a0f0-f2683b10fa89 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:17.463187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c9a65-c455-4eaf-a0f0-f2683b10fa89 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:17.463559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c9a65-c455-4eaf-a0f0-f2683b10fa89 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:17.466939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:17.474508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c9a65-c455-4eaf-a0f0-f2683b10fa89 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:17.477273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127c9a65-c455-4eaf-a0f0-f2683b10fa89 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-127c9a65-c455-4eaf-a0f0-f2683b10fa89 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:55:17.496629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:17.497696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:17.498645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 625/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:17.504801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc9da7-b5a6-4ea5-bc76-819e4a80ec59 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:17.608892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc9da7-b5a6-4ea5-bc76-819e4a80ec59 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:17.651254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f44dd-2d3c-4dcf-b020-e5562d6fa0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:17.652006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f44dd-2d3c-4dcf-b020-e5562d6fa0e0 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:17.652316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f44dd-2d3c-4dcf-b020-e5562d6fa0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:17.652519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547f44dd-2d3c-4dcf-b020-e5562d6fa0e0 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:17.653491 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f44dd-2d3c-4dcf-b020-e5562d6fa0e0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:17.653768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f44dd-2d3c-4dcf-b020-e5562d6fa0e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:17.655385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547f44dd-2d3c-4dcf-b020-e5562d6fa0e0 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:17.675354 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 615/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Sun Jun 26 02:55:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:17.681323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c9a65-c455-4eaf-a0f0-f2683b10fa89 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:17.793914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c9a65-c455-4eaf-a0f0-f2683b10fa89 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:17.830230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3209b-3d5a-463c-b734-5cdc2b8d0f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:17.830920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3209b-3d5a-463c-b734-5cdc2b8d0f89 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:17.831204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3209b-3d5a-463c-b734-5cdc2b8d0f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:17.831499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd3209b-3d5a-463c-b734-5cdc2b8d0f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:17.901200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:17.909820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3209b-3d5a-463c-b734-5cdc2b8d0f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:17.911693 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd3209b-3d5a-463c-b734-5cdc2b8d0f89 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:18.018177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 626/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:55:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:55:18.026960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f44dd-2d3c-4dcf-b020-e5562d6fa0e0 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:18.055840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:18.133910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547f44dd-2d3c-4dcf-b020-e5562d6fa0e0 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:18.172846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7579783-ac0c-4c41-920b-cb4348f1183e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:18.173597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7579783-ac0c-4c41-920b-cb4348f1183e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:18.173857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7579783-ac0c-4c41-920b-cb4348f1183e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:18.174164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7579783-ac0c-4c41-920b-cb4348f1183e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:18.185028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7579783-ac0c-4c41-920b-cb4348f1183e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:18.186502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7579783-ac0c-4c41-920b-cb4348f1183e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-f7579783-ac0c-4c41-920b-cb4348f1183e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:18.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:18.207064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:18.208059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:18.208059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:18.208059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:18.208059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:18.208059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 616/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:18.212953 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3209b-3d5a-463c-b734-5cdc2b8d0f89 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:18.309605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:18.315821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd3209b-3d5a-463c-b734-5cdc2b8d0f89 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:18.351850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b1cc0-525e-4112-a890-adfa4bd17385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:18.352510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b1cc0-525e-4112-a890-adfa4bd17385 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:18.352808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b1cc0-525e-4112-a890-adfa4bd17385 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:18.353067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b1cc0-525e-4112-a890-adfa4bd17385 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:18.353789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b1cc0-525e-4112-a890-adfa4bd17385 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:18.354125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b1cc0-525e-4112-a890-adfa4bd17385 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:18.355486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b1cc0-525e-4112-a890-adfa4bd17385 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:18.367292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 627/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:18.373186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7579783-ac0c-4c41-920b-cb4348f1183e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:18.465444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:18.490807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7579783-ac0c-4c41-920b-cb4348f1183e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:18.531836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14ea6c-fdfa-462a-b336-ab1cf3424e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:18.532795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14ea6c-fdfa-462a-b336-ab1cf3424e78 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:18.533084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14ea6c-fdfa-462a-b336-ab1cf3424e78 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:18.533360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff14ea6c-fdfa-462a-b336-ab1cf3424e78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:18.540832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14ea6c-fdfa-462a-b336-ab1cf3424e78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:18.542321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff14ea6c-fdfa-462a-b336-ab1cf3424e78 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-ff14ea6c-fdfa-462a-b336-ab1cf3424e78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:55:18.558668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:18.559667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:18.560688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:55:18.560688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:55:18.560688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:55:18.560688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:55:18.560688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:55:18.560688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:55:18.560688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:18.560688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:18.560688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 617/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:55:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:18.566683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b1cc0-525e-4112-a890-adfa4bd17385 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:18.688651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b1cc0-525e-4112-a890-adfa4bd17385 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:18.726460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc9895a-1566-4efa-82e6-fe711fc833a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:18.727027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9895a-1566-4efa-82e6-fe711fc833a6 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:18.727378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9895a-1566-4efa-82e6-fe711fc833a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:18.727657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc9895a-1566-4efa-82e6-fe711fc833a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:18.728434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9895a-1566-4efa-82e6-fe711fc833a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:18.728740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9895a-1566-4efa-82e6-fe711fc833a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:18.730252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc9895a-1566-4efa-82e6-fe711fc833a6 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:18.744544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 628/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:55:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:18.750996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14ea6c-fdfa-462a-b336-ab1cf3424e78 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:18.864632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff14ea6c-fdfa-462a-b336-ab1cf3424e78 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:18.904790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76caa128-7fa5-41ef-8f64-b63f20a1eced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:18.905449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa128-7fa5-41ef-8f64-b63f20a1eced None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:18.905732 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa128-7fa5-41ef-8f64-b63f20a1eced None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:18.906014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76caa128-7fa5-41ef-8f64-b63f20a1eced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:18.906869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76caa128-7fa5-41ef-8f64-b63f20a1eced None admin] RBAC: Adding query filter params (role.id=10b1c5eed2754da5adfd5217256adba8, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=19ce6f259c22441486bfc04811a271b8) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:18.907182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76caa128-7fa5-41ef-8f64-b63f20a1eced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:18.908594 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76caa128-7fa5-41ef-8f64-b63f20a1eced None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:18.915715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 618/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:55:18 2022] GET /identity/v3/role_assignments?user.id=19ce6f259c22441486bfc04811a271b8&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=10b1c5eed2754da5adfd5217256adba8 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:21.211821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd419b0-4f9b-47ea-8497-a35394747a26 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:21.212376 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd419b0-4f9b-47ea-8497-a35394747a26 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:21.212472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd419b0-4f9b-47ea-8497-a35394747a26 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:21.214075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 629/1247] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:21.219444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3be86-b35a-4341-a3b5-381d2215491b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:21.219924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3be86-b35a-4341-a3b5-381d2215491b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:21.220019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f3be86-b35a-4341-a3b5-381d2215491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:21.225051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:21.239007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e5f3be86-b35a-4341-a3b5-381d2215491b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:21.240374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e5f3be86-b35a-4341-a3b5-381d2215491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:21.310839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 619/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:21.338399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd419b0-4f9b-47ea-8497-a35394747a26 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:21.358377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:21.451710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd419b0-4f9b-47ea-8497-a35394747a26 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:21.492494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186f3bd-26c7-43f6-90b1-2e22082b05e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:21.493244 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186f3bd-26c7-43f6-90b1-2e22082b05e8 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:21.493539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186f3bd-26c7-43f6-90b1-2e22082b05e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:21.493799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186f3bd-26c7-43f6-90b1-2e22082b05e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:21.507339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0186f3bd-26c7-43f6-90b1-2e22082b05e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-0186f3bd-26c7-43f6-90b1-2e22082b05e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:55:21.516149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:21.517282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:21.518454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 630/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:55:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:21.525127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3be86-b35a-4341-a3b5-381d2215491b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:21.646456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f3be86-b35a-4341-a3b5-381d2215491b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:21.683138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:21.689710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c65f0-d18a-4db8-a694-53e77ab35b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:21.690477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c65f0-d18a-4db8-a694-53e77ab35b8d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:21.690719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c65f0-d18a-4db8-a694-53e77ab35b8d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:21.691020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598c65f0-d18a-4db8-a694-53e77ab35b8d None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:21.691819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c65f0-d18a-4db8-a694-53e77ab35b8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:21.692134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c65f0-d18a-4db8-a694-53e77ab35b8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:21.693590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598c65f0-d18a-4db8-a694-53e77ab35b8d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:21.704914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 620/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:55:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:21.711439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186f3bd-26c7-43f6-90b1-2e22082b05e8 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:21.796834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:21.823913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186f3bd-26c7-43f6-90b1-2e22082b05e8 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:21.866959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fdb60-1aec-43d2-8011-5309afe67217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:21.867955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fdb60-1aec-43d2-8011-5309afe67217 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:21.868237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fdb60-1aec-43d2-8011-5309afe67217 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:21.868523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709fdb60-1aec-43d2-8011-5309afe67217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:21.882440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709fdb60-1aec-43d2-8011-5309afe67217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:21.883954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709fdb60-1aec-43d2-8011-5309afe67217 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-709fdb60-1aec-43d2-8011-5309afe67217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:21.903672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:21.904756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:21.905634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:21.905634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:21.905634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:21.905634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:21.905634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 631/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:21.910915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c65f0-d18a-4db8-a694-53e77ab35b8d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:22.024474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598c65f0-d18a-4db8-a694-53e77ab35b8d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:22.063585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497e141-97f6-4b71-b284-41ba476de66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:22.064150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497e141-97f6-4b71-b284-41ba476de66e None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:22.064432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497e141-97f6-4b71-b284-41ba476de66e None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:22.064707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497e141-97f6-4b71-b284-41ba476de66e None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:22.065424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497e141-97f6-4b71-b284-41ba476de66e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:22.065731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497e141-97f6-4b71-b284-41ba476de66e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:22.067119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497e141-97f6-4b71-b284-41ba476de66e None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:22.079646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 621/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:55:22.085365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fdb60-1aec-43d2-8011-5309afe67217 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:22.205280 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709fdb60-1aec-43d2-8011-5309afe67217 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:22.237786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:22.248122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995caa5-cd50-49ee-ad5e-b7f95736ac88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:22.248929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995caa5-cd50-49ee-ad5e-b7f95736ac88 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:22.249455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995caa5-cd50-49ee-ad5e-b7f95736ac88 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:22.249877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995caa5-cd50-49ee-ad5e-b7f95736ac88 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:22.259985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995caa5-cd50-49ee-ad5e-b7f95736ac88 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:22.261495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995caa5-cd50-49ee-ad5e-b7f95736ac88 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e995caa5-cd50-49ee-ad5e-b7f95736ac88 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:55:22.277022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:22.278289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:22.279770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 632/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:22.284839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497e141-97f6-4b71-b284-41ba476de66e None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:22.344274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:22.395091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497e141-97f6-4b71-b284-41ba476de66e None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:22.437203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16604b0a-77fc-4136-ab26-2590b1ba9fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:22.437874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16604b0a-77fc-4136-ab26-2590b1ba9fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:22.438199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16604b0a-77fc-4136-ab26-2590b1ba9fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:22.438604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16604b0a-77fc-4136-ab26-2590b1ba9fa8 None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:22.439803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16604b0a-77fc-4136-ab26-2590b1ba9fa8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:22.440216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16604b0a-77fc-4136-ab26-2590b1ba9fa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:22.441972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16604b0a-77fc-4136-ab26-2590b1ba9fa8 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:22.462966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 622/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Sun Jun 26 02:55:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:22.468776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995caa5-cd50-49ee-ad5e-b7f95736ac88 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:22.579106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995caa5-cd50-49ee-ad5e-b7f95736ac88 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:22.621306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fe5d8-7363-45d5-aaa6-866096dd9767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:22.622373 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fe5d8-7363-45d5-aaa6-866096dd9767 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:22.622657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fe5d8-7363-45d5-aaa6-866096dd9767 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:22.623068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79fe5d8-7363-45d5-aaa6-866096dd9767 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:22.688968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:22.704071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fe5d8-7363-45d5-aaa6-866096dd9767 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:22.705892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79fe5d8-7363-45d5-aaa6-866096dd9767 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:22.813635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 633/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:55:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:55:22.820188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16604b0a-77fc-4136-ab26-2590b1ba9fa8 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:22.938023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16604b0a-77fc-4136-ab26-2590b1ba9fa8 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:22.951423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:22.974187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b01b-6cc6-44d6-abbd-4c2bf996563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:22.975067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b01b-6cc6-44d6-abbd-4c2bf996563f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:22.976002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b01b-6cc6-44d6-abbd-4c2bf996563f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:22.976321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7502b01b-6cc6-44d6-abbd-4c2bf996563f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:22.987119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502b01b-6cc6-44d6-abbd-4c2bf996563f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:22.988734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7502b01b-6cc6-44d6-abbd-4c2bf996563f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-7502b01b-6cc6-44d6-abbd-4c2bf996563f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:23.007388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:23.008662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:23.014965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:23.014965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:23.014965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:23.014965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:23.014965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 623/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:23.015271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fe5d8-7363-45d5-aaa6-866096dd9767 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:23.110613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:23.125025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79fe5d8-7363-45d5-aaa6-866096dd9767 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:23.163065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268446dd-fb69-4908-933d-80d0382cc4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:23.163827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268446dd-fb69-4908-933d-80d0382cc4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:23.164194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268446dd-fb69-4908-933d-80d0382cc4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:23.164635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268446dd-fb69-4908-933d-80d0382cc4d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:23.165850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268446dd-fb69-4908-933d-80d0382cc4d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:23.166393 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268446dd-fb69-4908-933d-80d0382cc4d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:23.168701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268446dd-fb69-4908-933d-80d0382cc4d9 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:23.184971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 634/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:23.191512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b01b-6cc6-44d6-abbd-4c2bf996563f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:23.308089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7502b01b-6cc6-44d6-abbd-4c2bf996563f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:23.347597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684b28-662a-44c9-b02f-3e55515f3939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:23.348519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684b28-662a-44c9-b02f-3e55515f3939 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:23.348805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684b28-662a-44c9-b02f-3e55515f3939 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:23.349202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1684b28-662a-44c9-b02f-3e55515f3939 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:23.357558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1684b28-662a-44c9-b02f-3e55515f3939 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:23.359142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1684b28-662a-44c9-b02f-3e55515f3939 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-b1684b28-662a-44c9-b02f-3e55515f3939 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:55:23.375393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:23.376385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:23.377374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:55:23.377374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:55:23.377374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:55:23.377374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:55:23.377374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:55:23.377374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:55:23.377374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:23.377374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:23.377374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 624/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:23.383430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268446dd-fb69-4908-933d-80d0382cc4d9 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:23.496557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268446dd-fb69-4908-933d-80d0382cc4d9 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:23.533955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78635828-0001-4ee4-b997-96b2649ebcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:23.534822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635828-0001-4ee4-b997-96b2649ebcf5 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:23.535251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635828-0001-4ee4-b997-96b2649ebcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:23.535663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635828-0001-4ee4-b997-96b2649ebcf5 None admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:23.536585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78635828-0001-4ee4-b997-96b2649ebcf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:23.537069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78635828-0001-4ee4-b997-96b2649ebcf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:23.539134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78635828-0001-4ee4-b997-96b2649ebcf5 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:23.552958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 635/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:55:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:23.559788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684b28-662a-44c9-b02f-3e55515f3939 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:23.587030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:23.679076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1684b28-662a-44c9-b02f-3e55515f3939 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:23.714219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afac335-6ee6-4d42-8755-15b7ce49ecbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:23.714900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac335-6ee6-4d42-8755-15b7ce49ecbc None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:23.715244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac335-6ee6-4d42-8755-15b7ce49ecbc None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:23.715526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afac335-6ee6-4d42-8755-15b7ce49ecbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:23.716381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afac335-6ee6-4d42-8755-15b7ce49ecbc None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=19ce6f259c22441486bfc04811a271b8) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:23.716698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afac335-6ee6-4d42-8755-15b7ce49ecbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:23.718197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afac335-6ee6-4d42-8755-15b7ce49ecbc None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:23.724578 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 625/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:55:23 2022] GET /identity/v3/role_assignments?user.id=19ce6f259c22441486bfc04811a271b8&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:25.717789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98735a52-ff30-4edf-ba87-b60ddc02f1d9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:25.718359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98735a52-ff30-4edf-ba87-b60ddc02f1d9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:25.718487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98735a52-ff30-4edf-ba87-b60ddc02f1d9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:25.719883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 636/1261] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:25.725021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d3ae-78cd-4689-b12c-b4545dc96c19 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:25.725983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d3ae-78cd-4689-b12c-b4545dc96c19 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:25.726243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86d3ae-78cd-4689-b12c-b4545dc96c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:25.742609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5b86d3ae-78cd-4689-b12c-b4545dc96c19 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:25.743780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5b86d3ae-78cd-4689-b12c-b4545dc96c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:25.807533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 626/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:25.829852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98735a52-ff30-4edf-ba87-b60ddc02f1d9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:25.840964 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:25.931804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98735a52-ff30-4edf-ba87-b60ddc02f1d9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:25.964561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2f3d2-323b-498c-a7e2-4081e1b13bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:25.965343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2f3d2-323b-498c-a7e2-4081e1b13bda None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:25.965624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2f3d2-323b-498c-a7e2-4081e1b13bda None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:25.965899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2f3d2-323b-498c-a7e2-4081e1b13bda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:25.976226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f2f3d2-323b-498c-a7e2-4081e1b13bda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-e5f2f3d2-323b-498c-a7e2-4081e1b13bda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:55:25.983678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:25.984736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:25.985670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 637/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:55:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:25.991450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d3ae-78cd-4689-b12c-b4545dc96c19 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:26.009713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:26.086142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86d3ae-78cd-4689-b12c-b4545dc96c19 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:26.118582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a2c2e-51df-49bc-a40e-a106d3918c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:26.119235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a2c2e-51df-49bc-a40e-a106d3918c2f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:26.119515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a2c2e-51df-49bc-a40e-a106d3918c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:26.119788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3a2c2e-51df-49bc-a40e-a106d3918c2f None admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:26.120519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a2c2e-51df-49bc-a40e-a106d3918c2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:26.120824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a2c2e-51df-49bc-a40e-a106d3918c2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:26.122190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3a2c2e-51df-49bc-a40e-a106d3918c2f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:26.130182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 627/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:55:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:26.135586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2f3d2-323b-498c-a7e2-4081e1b13bda None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:26.200191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:26.232751 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2f3d2-323b-498c-a7e2-4081e1b13bda None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:26.265058 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999f35f-03a1-4726-bf32-2b94d7e36d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:26.265824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999f35f-03a1-4726-bf32-2b94d7e36d97 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:26.266122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999f35f-03a1-4726-bf32-2b94d7e36d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:26.266397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999f35f-03a1-4726-bf32-2b94d7e36d97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:26.276484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999f35f-03a1-4726-bf32-2b94d7e36d97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:26.277899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5999f35f-03a1-4726-bf32-2b94d7e36d97 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-5999f35f-03a1-4726-bf32-2b94d7e36d97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:26.294689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:26.295708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:26.296788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:26.296788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:26.296788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:26.296788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:26.296788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 638/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:26.301419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a2c2e-51df-49bc-a40e-a106d3918c2f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:26.380274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:26.399180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3a2c2e-51df-49bc-a40e-a106d3918c2f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:26.430712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fa32f-99f1-451b-9dc3-53671ad3ba64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:26.431350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fa32f-99f1-451b-9dc3-53671ad3ba64 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:26.431630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fa32f-99f1-451b-9dc3-53671ad3ba64 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:26.431902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7fa32f-99f1-451b-9dc3-53671ad3ba64 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:26.432608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fa32f-99f1-451b-9dc3-53671ad3ba64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:26.432913 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fa32f-99f1-451b-9dc3-53671ad3ba64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:26.434288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7fa32f-99f1-451b-9dc3-53671ad3ba64 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:26.445547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 628/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:26.450947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999f35f-03a1-4726-bf32-2b94d7e36d97 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:26.546559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999f35f-03a1-4726-bf32-2b94d7e36d97 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:26.564899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:26.578638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ac9fa-7d77-41bf-ba97-b3ec553a5507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:26.579412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ac9fa-7d77-41bf-ba97-b3ec553a5507 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:26.579692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ac9fa-7d77-41bf-ba97-b3ec553a5507 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:26.579966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008ac9fa-7d77-41bf-ba97-b3ec553a5507 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:26.587606 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ac9fa-7d77-41bf-ba97-b3ec553a5507 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:26.589015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008ac9fa-7d77-41bf-ba97-b3ec553a5507 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-008ac9fa-7d77-41bf-ba97-b3ec553a5507 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:55:26.603212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:26.604249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:26.605459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:26.605459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:26.605459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:55:26.605459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:55:26.605459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:55:26.605459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:55:26.605459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:55:26.605459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:55:26.605459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:26.605459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:26.610180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 639/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:26.611117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fa32f-99f1-451b-9dc3-53671ad3ba64 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:26.719442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7fa32f-99f1-451b-9dc3-53671ad3ba64 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:26.752116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150c58a-0037-4636-bcde-cc7f5724aaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:26.752776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150c58a-0037-4636-bcde-cc7f5724aaca None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:26.753057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150c58a-0037-4636-bcde-cc7f5724aaca None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:26.753330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d150c58a-0037-4636-bcde-cc7f5724aaca None admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:26.754182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150c58a-0037-4636-bcde-cc7f5724aaca None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:26.754494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150c58a-0037-4636-bcde-cc7f5724aaca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:26.755903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d150c58a-0037-4636-bcde-cc7f5724aaca None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:26.764696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:26.773457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 629/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Sun Jun 26 02:55:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:55:26.779087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ac9fa-7d77-41bf-ba97-b3ec553a5507 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:26.875120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008ac9fa-7d77-41bf-ba97-b3ec553a5507 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:26.907762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd878035-b711-4495-8f84-1898b0f8b598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:26.908539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd878035-b711-4495-8f84-1898b0f8b598 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:26.908820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd878035-b711-4495-8f84-1898b0f8b598 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:26.909090 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd878035-b711-4495-8f84-1898b0f8b598 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:26.919308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd878035-b711-4495-8f84-1898b0f8b598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:26.920736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd878035-b711-4495-8f84-1898b0f8b598 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-fd878035-b711-4495-8f84-1898b0f8b598 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:26.936973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:26.937998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:26.938938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:26.938938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:26.938938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:26.938938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:26.938938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 640/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:26.943787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150c58a-0037-4636-bcde-cc7f5724aaca None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:27.040027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d150c58a-0037-4636-bcde-cc7f5724aaca None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:27.072453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce26927-f727-4768-a7bd-b6aba82bd96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:27.073103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce26927-f727-4768-a7bd-b6aba82bd96b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:27.073390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce26927-f727-4768-a7bd-b6aba82bd96b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:27.073667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce26927-f727-4768-a7bd-b6aba82bd96b None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:27.074408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce26927-f727-4768-a7bd-b6aba82bd96b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:27.074722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce26927-f727-4768-a7bd-b6aba82bd96b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:27.076071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce26927-f727-4768-a7bd-b6aba82bd96b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:27.087501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 630/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:27.092906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd878035-b711-4495-8f84-1898b0f8b598 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:27.122972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:27.189003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd878035-b711-4495-8f84-1898b0f8b598 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:27.220881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e954dd-4ee9-4c4c-9ad6-6358718340e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:27.221647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e954dd-4ee9-4c4c-9ad6-6358718340e3 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:27.221926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e954dd-4ee9-4c4c-9ad6-6358718340e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:27.222229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e954dd-4ee9-4c4c-9ad6-6358718340e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:27.228603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e954dd-4ee9-4c4c-9ad6-6358718340e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:27.230016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e954dd-4ee9-4c4c-9ad6-6358718340e3 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-62e954dd-4ee9-4c4c-9ad6-6358718340e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:55:27.244125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:27.245149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:27.246355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:55:27.246355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:55:27.246355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:55:27.246355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:55:27.246355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:55:27.246355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:55:27.246355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:27.246355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:27.246355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 641/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:55:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:27.251199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce26927-f727-4768-a7bd-b6aba82bd96b None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:27.307753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:27.348426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce26927-f727-4768-a7bd-b6aba82bd96b None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:27.382456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a877db-1cc6-4b37-864c-e19ea51e4d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:27.383050 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a877db-1cc6-4b37-864c-e19ea51e4d9a None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:27.383333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a877db-1cc6-4b37-864c-e19ea51e4d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:27.383611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a877db-1cc6-4b37-864c-e19ea51e4d9a None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:27.384391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a877db-1cc6-4b37-864c-e19ea51e4d9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:27.384704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a877db-1cc6-4b37-864c-e19ea51e4d9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:27.386147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a877db-1cc6-4b37-864c-e19ea51e4d9a None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:27.397808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 631/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:55:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:27.403519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e954dd-4ee9-4c4c-9ad6-6358718340e3 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:27.491746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:27.500881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e954dd-4ee9-4c4c-9ad6-6358718340e3 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:27.533396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0416d45-070a-485a-a52e-97b315381504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:27.534281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416d45-070a-485a-a52e-97b315381504 None admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:27.534557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416d45-070a-485a-a52e-97b315381504 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:27.534825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0416d45-070a-485a-a52e-97b315381504 None admin] PATH_INFO: `/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/19ce6f259c22441486bfc04811a271b8/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:27.536732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0416d45-070a-485a-a52e-97b315381504 None admin] RBAC: Authorizing `identity:create_grant(project_id=33208982d2b04888aa7d4e1ca0d5df53, user_id=19ce6f259c22441486bfc04811a271b8, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:27.538684 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0416d45-070a-485a-a52e-97b315381504 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:27.550638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 642/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Jun 26 02:55:27 2022] PUT /identity/v3/projects/33208982d2b04888aa7d4e1ca0d5df53/users/19ce6f259c22441486bfc04811a271b8/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 02:55:29.541607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be5c83-c7a6-4e7a-862c-56a552544be1 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:29.542179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be5c83-c7a6-4e7a-862c-56a552544be1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:29.542179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67be5c83-c7a6-4e7a-862c-56a552544be1 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:29.543577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 632/1274] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:29.548711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629dfe1-2322-41c8-9568-ae7cb24a36f5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:29.549074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629dfe1-2322-41c8-9568-ae7cb24a36f5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:29.549262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629dfe1-2322-41c8-9568-ae7cb24a36f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:29.565407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8629dfe1-2322-41c8-9568-ae7cb24a36f5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:29.566615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8629dfe1-2322-41c8-9568-ae7cb24a36f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:29.640056 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 643/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:29.666772 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be5c83-c7a6-4e7a-862c-56a552544be1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:29.763071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67be5c83-c7a6-4e7a-862c-56a552544be1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:29.780611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:29.796747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdffd9-92bc-453d-bb7c-8b2f50648ca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:29.797455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdffd9-92bc-453d-bb7c-8b2f50648ca2 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:29.797733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdffd9-92bc-453d-bb7c-8b2f50648ca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:29.798003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fdffd9-92bc-453d-bb7c-8b2f50648ca2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:29.807590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fdffd9-92bc-453d-bb7c-8b2f50648ca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-d3fdffd9-92bc-453d-bb7c-8b2f50648ca2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 26 02:55:29.815007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:29.816112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:29.817290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 633/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Jun 26 02:55:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:29.822480 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629dfe1-2322-41c8-9568-ae7cb24a36f5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:29.900278 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:29.918718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629dfe1-2322-41c8-9568-ae7cb24a36f5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:29.951562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bba4c2-72de-4760-9e0f-76cb8e3aadba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:29.952217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bba4c2-72de-4760-9e0f-76cb8e3aadba None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:29.952495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bba4c2-72de-4760-9e0f-76cb8e3aadba None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:29.952765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bba4c2-72de-4760-9e0f-76cb8e3aadba None admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:29.953497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bba4c2-72de-4760-9e0f-76cb8e3aadba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:29.953803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bba4c2-72de-4760-9e0f-76cb8e3aadba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:29.955183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bba4c2-72de-4760-9e0f-76cb8e3aadba None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:29.963791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 644/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Jun 26 02:55:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:29.969465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdffd9-92bc-453d-bb7c-8b2f50648ca2 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:30.064122 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fdffd9-92bc-453d-bb7c-8b2f50648ca2 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:30.096208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e95947-1bc3-49c6-b788-f2fc41ed8e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:30.096965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95947-1bc3-49c6-b788-f2fc41ed8e5f None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:30.097243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95947-1bc3-49c6-b788-f2fc41ed8e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:30.097510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e95947-1bc3-49c6-b788-f2fc41ed8e5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:30.107458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e95947-1bc3-49c6-b788-f2fc41ed8e5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:30.108879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e95947-1bc3-49c6-b788-f2fc41ed8e5f None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-a1e95947-1bc3-49c6-b788-f2fc41ed8e5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:30.125300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:30.126353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:30.131141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:30.131141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:30.131141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:30.131141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:30.131141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 634/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 26 02:55:30.132447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bba4c2-72de-4760-9e0f-76cb8e3aadba None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:30.228414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bba4c2-72de-4760-9e0f-76cb8e3aadba None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:30.261577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97556409-7008-41cf-9c74-e28304d0643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:30.262253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97556409-7008-41cf-9c74-e28304d0643a None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:30.262535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97556409-7008-41cf-9c74-e28304d0643a None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:30.262807 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97556409-7008-41cf-9c74-e28304d0643a None admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:30.263523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97556409-7008-41cf-9c74-e28304d0643a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:30.263828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97556409-7008-41cf-9c74-e28304d0643a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:30.265164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97556409-7008-41cf-9c74-e28304d0643a None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:30.276532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 645/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:30.282004 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e95947-1bc3-49c6-b788-f2fc41ed8e5f None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:30.311779 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:30.378651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e95947-1bc3-49c6-b788-f2fc41ed8e5f None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:30.410518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e6ebf-bc43-46a1-b7d2-93af138c3d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:30.411290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e6ebf-bc43-46a1-b7d2-93af138c3d20 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:30.411572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e6ebf-bc43-46a1-b7d2-93af138c3d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:30.411843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e6ebf-bc43-46a1-b7d2-93af138c3d20 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:30.418978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e6ebf-bc43-46a1-b7d2-93af138c3d20 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:30.420388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e6ebf-bc43-46a1-b7d2-93af138c3d20 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-7c5e6ebf-bc43-46a1-b7d2-93af138c3d20 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 26 02:55:30.434703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:30.435823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:30.436981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 635/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:30.441435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97556409-7008-41cf-9c74-e28304d0643a None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:30.445728 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:30.536723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97556409-7008-41cf-9c74-e28304d0643a None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:30.569592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e49ce-740f-48eb-a4ac-6cf650a63933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:30.570281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e49ce-740f-48eb-a4ac-6cf650a63933 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:30.570561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e49ce-740f-48eb-a4ac-6cf650a63933 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:30.570832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5e49ce-740f-48eb-a4ac-6cf650a63933 None admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:30.571649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e49ce-740f-48eb-a4ac-6cf650a63933 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:30.571956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e49ce-740f-48eb-a4ac-6cf650a63933 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:30.573358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5e49ce-740f-48eb-a4ac-6cf650a63933 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:30.589842 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 646/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Sun Jun 26 02:55:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:30.595685 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e6ebf-bc43-46a1-b7d2-93af138c3d20 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:30.691233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:30.700715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e6ebf-bc43-46a1-b7d2-93af138c3d20 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:30.732549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c813fff-c29d-4c90-8bb1-e919d0927391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:30.733184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c813fff-c29d-4c90-8bb1-e919d0927391 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:30.733464 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c813fff-c29d-4c90-8bb1-e919d0927391 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:30.733736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c813fff-c29d-4c90-8bb1-e919d0927391 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:30.797259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c813fff-c29d-4c90-8bb1-e919d0927391 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:30.798714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c813fff-c29d-4c90-8bb1-e919d0927391 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:30.885960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:30.896327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 636/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Jun 26 02:55:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 26 02:55:30.904574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e49ce-740f-48eb-a4ac-6cf650a63933 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:30.965493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:31.003601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5e49ce-740f-48eb-a4ac-6cf650a63933 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:31.035675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b125b-1842-41b2-bd39-ec28b09dc1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:31.036450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b125b-1842-41b2-bd39-ec28b09dc1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:31.036727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b125b-1842-41b2-bd39-ec28b09dc1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:31.036995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2b125b-1842-41b2-bd39-ec28b09dc1e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:31.047116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b125b-1842-41b2-bd39-ec28b09dc1e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:31.048549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2b125b-1842-41b2-bd39-ec28b09dc1e0 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-6e2b125b-1842-41b2-bd39-ec28b09dc1e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 26 02:55:31.065449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:31.066503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:31.067420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 26 02:55:31.067420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 26 02:55:31.067420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 26 02:55:31.067420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:31.067420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 647/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:31.072422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c813fff-c29d-4c90-8bb1-e919d0927391 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:31.168303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c813fff-c29d-4c90-8bb1-e919d0927391 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:31.200241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4588a0-809a-4e6d-b019-0b75080be947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:31.200899 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4588a0-809a-4e6d-b019-0b75080be947 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:31.201184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4588a0-809a-4e6d-b019-0b75080be947 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:31.201462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4588a0-809a-4e6d-b019-0b75080be947 None admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:31.202333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4588a0-809a-4e6d-b019-0b75080be947 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:31.202643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4588a0-809a-4e6d-b019-0b75080be947 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:31.204014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec4588a0-809a-4e6d-b019-0b75080be947 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:31.216224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 637/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Jun 26 02:55:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:31.221779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b125b-1842-41b2-bd39-ec28b09dc1e0 None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:31.320640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2b125b-1842-41b2-bd39-ec28b09dc1e0 None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:31.334020 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:31.354605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0d00-b4ec-4840-b8f2-63e2d0445c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:31.355383 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0d00-b4ec-4840-b8f2-63e2d0445c6e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:31.355669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0d00-b4ec-4840-b8f2-63e2d0445c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:31.355946 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363d0d00-b4ec-4840-b8f2-63e2d0445c6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:31.362452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d0d00-b4ec-4840-b8f2-63e2d0445c6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:31.364395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363d0d00-b4ec-4840-b8f2-63e2d0445c6e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application [None req-363d0d00-b4ec-4840-b8f2-63e2d0445c6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 26 02:55:31.378119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:31.379163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:31.380222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 26 02:55:31.380222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 26 02:55:31.380222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 26 02:55:31.380222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 26 02:55:31.380222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 26 02:55:31.380222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 26 02:55:31.380222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 26 02:55:31.380222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: ERROR keystone.server.flask.application Jun 26 02:55:31.380222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 648/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Jun 26 02:55:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:31.385521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4588a0-809a-4e6d-b019-0b75080be947 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:31.434418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:31.482981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4588a0-809a-4e6d-b019-0b75080be947 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:31.515637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428028eb-66b2-49c5-9989-7f51a0643e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:31.516285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428028eb-66b2-49c5-9989-7f51a0643e2b None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:31.516568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428028eb-66b2-49c5-9989-7f51a0643e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:31.516840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428028eb-66b2-49c5-9989-7f51a0643e2b None admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:31.517617 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428028eb-66b2-49c5-9989-7f51a0643e2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:31.517926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428028eb-66b2-49c5-9989-7f51a0643e2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:31.519370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428028eb-66b2-49c5-9989-7f51a0643e2b None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:31.531771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 638/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Jun 26 02:55:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:31.537882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0d00-b4ec-4840-b8f2-63e2d0445c6e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:31.638449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363d0d00-b4ec-4840-b8f2-63e2d0445c6e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:31.670806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58214362-9a3b-4bb5-bb90-91a03dca57e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:31.671445 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58214362-9a3b-4bb5-bb90-91a03dca57e7 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:31.671727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58214362-9a3b-4bb5-bb90-91a03dca57e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:31.672003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58214362-9a3b-4bb5-bb90-91a03dca57e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:31.672822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58214362-9a3b-4bb5-bb90-91a03dca57e7 None admin] RBAC: Adding query filter params (role.id=0b41a54bd0dd41aaaf09a3a659d1f372, scope.project.id=33208982d2b04888aa7d4e1ca0d5df53, user.id=19ce6f259c22441486bfc04811a271b8) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:31.673133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58214362-9a3b-4bb5-bb90-91a03dca57e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:31.674587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58214362-9a3b-4bb5-bb90-91a03dca57e7 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:31.680818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 649/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Jun 26 02:55:31 2022] GET /identity/v3/role_assignments?user.id=19ce6f259c22441486bfc04811a271b8&scope.project.id=33208982d2b04888aa7d4e1ca0d5df53&role.id=0b41a54bd0dd41aaaf09a3a659d1f372 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:33.693440 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92858706-83f0-4e97-a724-72783bb20f44 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:33.694516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92858706-83f0-4e97-a724-72783bb20f44 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:33.694516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92858706-83f0-4e97-a724-72783bb20f44 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:33.695946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 639/1288] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:33.701095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ffc5-4d78-44de-9328-8df5afa33de4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:33.701499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ffc5-4d78-44de-9328-8df5afa33de4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:33.701648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9ffc5-4d78-44de-9328-8df5afa33de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:33.718613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-aee9ffc5-4d78-44de-9328-8df5afa33de4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:33.719781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aee9ffc5-4d78-44de-9328-8df5afa33de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:33.748045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:33.784782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 650/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:33.806655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92858706-83f0-4e97-a724-72783bb20f44 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:33.907128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92858706-83f0-4e97-a724-72783bb20f44 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:33.917421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:33.940683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f5a6f-81dd-480c-8b25-1afbc0cf0b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:33.941466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f5a6f-81dd-480c-8b25-1afbc0cf0b6d None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:33.941750 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f5a6f-81dd-480c-8b25-1afbc0cf0b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:33.942026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1f5a6f-81dd-480c-8b25-1afbc0cf0b6d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:33.942702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f5a6f-81dd-480c-8b25-1afbc0cf0b6d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:33.944736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1f5a6f-81dd-480c-8b25-1afbc0cf0b6d None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-4d1f5a6f-81dd-480c-8b25-1afbc0cf0b6d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:33.956134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:33.957342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:33.958411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 640/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Jun 26 02:55:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:33.963394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9ffc5-4d78-44de-9328-8df5afa33de4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:34.060284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9ffc5-4d78-44de-9328-8df5afa33de4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:34.093182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0441f76b-dee3-4dab-8eae-75349f796ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:34.093834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441f76b-dee3-4dab-8eae-75349f796ace None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:34.094146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441f76b-dee3-4dab-8eae-75349f796ace None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:34.094424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0441f76b-dee3-4dab-8eae-75349f796ace None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:34.095122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441f76b-dee3-4dab-8eae-75349f796ace None admin] RBAC: Adding query filter params (name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:34.095428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441f76b-dee3-4dab-8eae-75349f796ace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:34.096776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0441f76b-dee3-4dab-8eae-75349f796ace None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:34.101515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 651/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Sun Jun 26 02:55:33 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 02:55:34.107217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f5a6f-81dd-480c-8b25-1afbc0cf0b6d None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:34.203936 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1f5a6f-81dd-480c-8b25-1afbc0cf0b6d None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:34.235611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3127d018-ee0c-41ea-89b8-fe4cad2b9250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:34.236251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127d018-ee0c-41ea-89b8-fe4cad2b9250 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:34.236536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127d018-ee0c-41ea-89b8-fe4cad2b9250 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:34.236812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3127d018-ee0c-41ea-89b8-fe4cad2b9250 None admin] PATH_INFO: `/v3/services` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:34.237507 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127d018-ee0c-41ea-89b8-fe4cad2b9250 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:34.237814 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127d018-ee0c-41ea-89b8-fe4cad2b9250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:34.239200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3127d018-ee0c-41ea-89b8-fe4cad2b9250 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:34.244023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 641/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Sun Jun 26 02:55:34 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:36.228492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd22b44-2fb4-433b-8813-7d1397f6ff88 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:36.229013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd22b44-2fb4-433b-8813-7d1397f6ff88 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:36.229129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd22b44-2fb4-433b-8813-7d1397f6ff88 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:36.230481 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 652/1293] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:36.235649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ad765-642b-4615-b502-1cb521730b0f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:36.236063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ad765-642b-4615-b502-1cb521730b0f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:36.236201 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ad765-642b-4615-b502-1cb521730b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:36.252674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dd4ad765-642b-4615-b502-1cb521730b0f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:36.253856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dd4ad765-642b-4615-b502-1cb521730b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:36.282443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:36.318647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 642/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:36.341632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd22b44-2fb4-433b-8813-7d1397f6ff88 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:36.372112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:36.440102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd22b44-2fb4-433b-8813-7d1397f6ff88 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:36.472928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea79ff-12b6-4296-a5c4-37bdee172529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:36.473566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea79ff-12b6-4296-a5c4-37bdee172529 None admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:36.473843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea79ff-12b6-4296-a5c4-37bdee172529 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:36.474140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea79ff-12b6-4296-a5c4-37bdee172529 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:36.474854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea79ff-12b6-4296-a5c4-37bdee172529 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:36.476792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea79ff-12b6-4296-a5c4-37bdee172529 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:36.489445 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 653/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Jun 26 02:55:36 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 02:55:38.510870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a5565-1b0d-48ed-b6d4-31904b34e89b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:38.511444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a5565-1b0d-48ed-b6d4-31904b34e89b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:38.511444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059a5565-1b0d-48ed-b6d4-31904b34e89b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:38.512855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 643/1296] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:38.517994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25932c1e-2647-49e8-962b-f14271ff9d0c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:38.518411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25932c1e-2647-49e8-962b-f14271ff9d0c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:38.518579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25932c1e-2647-49e8-962b-f14271ff9d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:38.535228 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-25932c1e-2647-49e8-962b-f14271ff9d0c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:38.536408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-25932c1e-2647-49e8-962b-f14271ff9d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:38.606666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 654/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:38 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:38.632126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a5565-1b0d-48ed-b6d4-31904b34e89b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:38.745111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059a5565-1b0d-48ed-b6d4-31904b34e89b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:38.779188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540a683-c23a-4b47-a3cf-4340fc28e8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:38.779912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540a683-c23a-4b47-a3cf-4340fc28e8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:38.780196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540a683-c23a-4b47-a3cf-4340fc28e8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:38.780471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540a683-c23a-4b47-a3cf-4340fc28e8f7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:38.781116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540a683-c23a-4b47-a3cf-4340fc28e8f7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:38.782502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7540a683-c23a-4b47-a3cf-4340fc28e8f7 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-7540a683-c23a-4b47-a3cf-4340fc28e8f7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:38.795845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:38.797217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:38.798565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 644/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Jun 26 02:55:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:38.802988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25932c1e-2647-49e8-962b-f14271ff9d0c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:38.823126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:38.903225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25932c1e-2647-49e8-962b-f14271ff9d0c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:38.934831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00206c-9efe-4405-af2d-071929bb6d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:38.935481 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00206c-9efe-4405-af2d-071929bb6d4e None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:38.935762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00206c-9efe-4405-af2d-071929bb6d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:38.936033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00206c-9efe-4405-af2d-071929bb6d4e None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:38.936726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00206c-9efe-4405-af2d-071929bb6d4e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:38.937027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00206c-9efe-4405-af2d-071929bb6d4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:38.938392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00206c-9efe-4405-af2d-071929bb6d4e None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:38.943418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 655/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Sun Jun 26 02:55:38 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:38.949745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540a683-c23a-4b47-a3cf-4340fc28e8f7 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:38.980415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:39.046608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7540a683-c23a-4b47-a3cf-4340fc28e8f7 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:39.078426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0f5087-9057-4b53-8bb3-780d52247bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:39.079065 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f5087-9057-4b53-8bb3-780d52247bdb None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:39.079346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f5087-9057-4b53-8bb3-780d52247bdb None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:39.079619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0f5087-9057-4b53-8bb3-780d52247bdb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:39.080393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f5087-9057-4b53-8bb3-780d52247bdb None admin] RBAC: Adding query filter params (interface=public, service_id=5af9870dc18949279bbf4e007fcd8000, region_id=RegionOne) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:39.080699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f5087-9057-4b53-8bb3-780d52247bdb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:39.082032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0f5087-9057-4b53-8bb3-780d52247bdb None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:39.087518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 645/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Jun 26 02:55:38 2022] GET /identity/v3/endpoints?service_id=5af9870dc18949279bbf4e007fcd8000&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:39.093184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00206c-9efe-4405-af2d-071929bb6d4e None admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:39.178180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:39.197186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00206c-9efe-4405-af2d-071929bb6d4e None admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:39.229353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4294f2-501c-427e-bd04-ffb993bc0780 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:39.230012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4294f2-501c-427e-bd04-ffb993bc0780 None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:39.230326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4294f2-501c-427e-bd04-ffb993bc0780 None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:39.230602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4294f2-501c-427e-bd04-ffb993bc0780 None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:39.231275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4294f2-501c-427e-bd04-ffb993bc0780 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:39.232626 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4294f2-501c-427e-bd04-ffb993bc0780 None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:39.239632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 656/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Jun 26 02:55:39 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 02:55:41.234389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cb0ae-2e9b-4de0-88b7-c0f9480b19f6 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:41.234941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cb0ae-2e9b-4de0-88b7-c0f9480b19f6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:41.234941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3cb0ae-2e9b-4de0-88b7-c0f9480b19f6 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:41.236349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 646/1302] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:55:41.241650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608e701-424d-489c-b739-37f601d38949 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:41.242026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608e701-424d-489c-b739-37f601d38949 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:41.242258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5608e701-424d-489c-b739-37f601d38949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:41.259261 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5608e701-424d-489c-b739-37f601d38949 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:55:41.260446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5608e701-424d-489c-b739-37f601d38949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:55:41.324817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 657/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:55:41 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 26 02:55:41.348473 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cb0ae-2e9b-4de0-88b7-c0f9480b19f6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:41.439869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:41.447454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3cb0ae-2e9b-4de0-88b7-c0f9480b19f6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:41.480408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731e125-0218-4314-b680-baebace061b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:41.481183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731e125-0218-4314-b680-baebace061b9 None admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:41.481464 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731e125-0218-4314-b680-baebace061b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:41.481734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731e125-0218-4314-b680-baebace061b9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:41.482402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731e125-0218-4314-b680-baebace061b9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:41.483746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731e125-0218-4314-b680-baebace061b9 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application [None req-d731e125-0218-4314-b680-baebace061b9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 26 02:55:41.496078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 26 02:55:41.497052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.get_or_create( Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application with Lock( Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self._enter() Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application return self.creator() Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application created_value = creator( Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: ERROR keystone.server.flask.application Jun 26 02:55:41.497890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 647/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Jun 26 02:55:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 26 02:55:41.503468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608e701-424d-489c-b739-37f601d38949 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:41.577240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:41.599501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5608e701-424d-489c-b739-37f601d38949 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:41.639380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07b3d4-e960-432a-b26f-1297ee4d1bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:41.640042 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07b3d4-e960-432a-b26f-1297ee4d1bcc None admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:41.640323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07b3d4-e960-432a-b26f-1297ee4d1bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:41.640597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07b3d4-e960-432a-b26f-1297ee4d1bcc None admin] PATH_INFO: `/v3/services` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:41.641291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07b3d4-e960-432a-b26f-1297ee4d1bcc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 02:55:41.641595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07b3d4-e960-432a-b26f-1297ee4d1bcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:41.642976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07b3d4-e960-432a-b26f-1297ee4d1bcc None admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:41.648060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 658/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Sun Jun 26 02:55:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 02:55:41.653758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731e125-0218-4314-b680-baebace061b9 None admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:55:41.748892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731e125-0218-4314-b680-baebace061b9 None admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:55:41.780234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3b57a-6c9d-4064-a308-7ba3da0fc665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:55:41.780863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3b57a-6c9d-4064-a308-7ba3da0fc665 None admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:55:41.781145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3b57a-6c9d-4064-a308-7ba3da0fc665 None admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:55:41.781417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3b57a-6c9d-4064-a308-7ba3da0fc665 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:55:41.782163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3b57a-6c9d-4064-a308-7ba3da0fc665 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:55:41.783503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb3b57a-6c9d-4064-a308-7ba3da0fc665 None admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:55:41.801383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 648/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Jun 26 02:55:41 2022] POST /identity/v3/endpoints => generated 331 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 02:55:51.645054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=30 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:55:51.797151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:44.311674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910ffe2-bfa8-4b37-9306-cd977fdb4809 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:44.312247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910ffe2-bfa8-4b37-9306-cd977fdb4809 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:44.312247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e910ffe2-bfa8-4b37-9306-cd977fdb4809 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:44.317588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 659/1307] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:56:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:56:44.319424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df025b82-d9f9-46d3-ab29-e8a0cd2d588d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:44.319877 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df025b82-d9f9-46d3-ab29-e8a0cd2d588d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:44.320079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df025b82-d9f9-46d3-ab29-e8a0cd2d588d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:44.392427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-df025b82-d9f9-46d3-ab29-e8a0cd2d588d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:56:44.393873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-df025b82-d9f9-46d3-ab29-e8a0cd2d588d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:44.564184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 649/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 02:56:44.623725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ada27d-a507-4326-a66c-4010c5b5852e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:44.624196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ada27d-a507-4326-a66c-4010c5b5852e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:44.624742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ada27d-a507-4326-a66c-4010c5b5852e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:44.626996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 660/1309] 10.222.0.13 () {54 vars in 964 bytes} [Sun Jun 26 02:56:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:56:44.636278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23322c0-e325-474e-a378-9b42f49b72b7 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:44.636636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23322c0-e325-474e-a378-9b42f49b72b7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:44.637003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23322c0-e325-474e-a378-9b42f49b72b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:44.723851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c23322c0-e325-474e-a378-9b42f49b72b7 None None] MFA Rules not processed for user `8ff708d6758d4f7b913e763824f3b791`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:56:44.725766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c23322c0-e325-474e-a378-9b42f49b72b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:44.750187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:44.834537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 650/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Sun Jun 26 02:56:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:56:44.835185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ada27d-a507-4326-a66c-4010c5b5852e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:56:44.840047 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a8ada27d-a507-4326-a66c-4010c5b5852e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:44.977071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ada27d-a507-4326-a66c-4010c5b5852e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:56:45.061479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3b2c5-76cc-4c80-a85f-59e405a7a01c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:56:45.062094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3b2c5-76cc-4c80-a85f-59e405a7a01c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:45.062370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3b2c5-76cc-4c80-a85f-59e405a7a01c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:45.062644 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3b2c5-76cc-4c80-a85f-59e405a7a01c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:45.064394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e7d3b2c5-76cc-4c80-a85f-59e405a7a01c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:45.121919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:45.183148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3b2c5-76cc-4c80-a85f-59e405a7a01c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:56:45.184741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3b2c5-76cc-4c80-a85f-59e405a7a01c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:56:45.220168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 661/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 02:56:47.554434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cf91a-29b7-42b2-835c-ffe4f2feab5f None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:47.554954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cf91a-29b7-42b2-835c-ffe4f2feab5f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:47.554954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6cf91a-29b7-42b2-835c-ffe4f2feab5f None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:47.556341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 651/1312] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:56:47.561699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d4e93-2391-49fa-b1da-daba40466137 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:47.562150 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d4e93-2391-49fa-b1da-daba40466137 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:47.562302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d4e93-2391-49fa-b1da-daba40466137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:47.643536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-757d4e93-2391-49fa-b1da-daba40466137 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:56:47.644733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-757d4e93-2391-49fa-b1da-daba40466137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:47.744072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 662/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:56:47.782949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cf91a-29b7-42b2-835c-ffe4f2feab5f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:56:47.784482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1d6cf91a-29b7-42b2-835c-ffe4f2feab5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:47.896192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6cf91a-29b7-42b2-835c-ffe4f2feab5f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:56:47.964643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:47.966894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2df96c-ee59-4804-a7ab-8c50af6b60d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:56:47.967529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2df96c-ee59-4804-a7ab-8c50af6b60d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:47.967822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2df96c-ee59-4804-a7ab-8c50af6b60d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:47.968340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2df96c-ee59-4804-a7ab-8c50af6b60d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:47.970511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3d2df96c-ee59-4804-a7ab-8c50af6b60d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:48.081857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2df96c-ee59-4804-a7ab-8c50af6b60d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:56:48.083372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2df96c-ee59-4804-a7ab-8c50af6b60d6 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:56:48.102992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 652/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:56:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:56:50.437290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8c986-fc64-4608-accb-1eaa0676df98 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:50.437805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8c986-fc64-4608-accb-1eaa0676df98 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:50.437900 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8c986-fc64-4608-accb-1eaa0676df98 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:50.439272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 663/1315] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:56:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:56:50.445028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc18d7f-648d-41f6-a52c-b1b70e25f528 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:50.445424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc18d7f-648d-41f6-a52c-b1b70e25f528 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:50.445581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc18d7f-648d-41f6-a52c-b1b70e25f528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:50.509316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1dc18d7f-648d-41f6-a52c-b1b70e25f528 None None] MFA Rules not processed for user `19c972280f644c03b032e16fcb039658`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:56:50.510553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1dc18d7f-648d-41f6-a52c-b1b70e25f528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:50.560179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:50.603453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 653/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:56:50 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:56:50.643655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8c986-fc64-4608-accb-1eaa0676df98 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:56:50.645185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-84f8c986-fc64-4608-accb-1eaa0676df98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:50.655071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:50.747594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f8c986-fc64-4608-accb-1eaa0676df98 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:56:50.822168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f3c85-42d7-461a-aa41-86a5b400b151 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:56:50.822803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f3c85-42d7-461a-aa41-86a5b400b151 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:50.823083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f3c85-42d7-461a-aa41-86a5b400b151 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:50.823351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f3c85-42d7-461a-aa41-86a5b400b151 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:50.824957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-891f3c85-42d7-461a-aa41-86a5b400b151 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:50.884891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:50.931573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f3c85-42d7-461a-aa41-86a5b400b151 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:56:50.932994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891f3c85-42d7-461a-aa41-86a5b400b151 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:56:50.950457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 664/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:56:50 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:56:53.493955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5500-6e90-4248-8463-96769d090466 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:53.494510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5500-6e90-4248-8463-96769d090466 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:53.494606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a5500-6e90-4248-8463-96769d090466 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:53.495937 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 654/1318] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:56:53.501227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f267620-68bb-4ab1-bed1-99994f19a4ed None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:53.501648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f267620-68bb-4ab1-bed1-99994f19a4ed None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:53.501797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f267620-68bb-4ab1-bed1-99994f19a4ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:53.564943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0f267620-68bb-4ab1-bed1-99994f19a4ed None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:56:53.566179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0f267620-68bb-4ab1-bed1-99994f19a4ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:53.665747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 665/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:56:53 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:56:53.704144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a5500-6e90-4248-8463-96769d090466 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:56:53.705675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ff7a5500-6e90-4248-8463-96769d090466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:53.809261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a5500-6e90-4248-8463-96769d090466 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:56:53.880950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452362ba-2a50-462f-bdac-f554333b3fa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:56:53.881597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452362ba-2a50-462f-bdac-f554333b3fa2 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:53.881880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452362ba-2a50-462f-bdac-f554333b3fa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:53.882195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452362ba-2a50-462f-bdac-f554333b3fa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:53.883826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-452362ba-2a50-462f-bdac-f554333b3fa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:53.891108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:53.989030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452362ba-2a50-462f-bdac-f554333b3fa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:56:53.991098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452362ba-2a50-462f-bdac-f554333b3fa2 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:56:54.008709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 655/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:56:53 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:56:56.677273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d3bd1-693d-4074-add3-1f3346ee6761 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:56.677841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d3bd1-693d-4074-add3-1f3346ee6761 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:56.677841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169d3bd1-693d-4074-add3-1f3346ee6761 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:56.679278 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 666/1321] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:56:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:56:56.684700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1925b9-d98b-4d5c-83c9-c5e4a1a22691 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:56.685151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1925b9-d98b-4d5c-83c9-c5e4a1a22691 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:56.685255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1925b9-d98b-4d5c-83c9-c5e4a1a22691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:56.749746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8d1925b9-d98b-4d5c-83c9-c5e4a1a22691 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:56:56.751024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8d1925b9-d98b-4d5c-83c9-c5e4a1a22691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:56.812256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:56.856210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 656/1322] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:56:56.892961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d3bd1-693d-4074-add3-1f3346ee6761 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:56:56.894767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-169d3bd1-693d-4074-add3-1f3346ee6761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:56.914970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:57.005309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169d3bd1-693d-4074-add3-1f3346ee6761 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:56:57.078327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ba9f18-d505-4c4e-89ec-903a5a1dbed2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:56:57.078982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba9f18-d505-4c4e-89ec-903a5a1dbed2 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:57.079266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba9f18-d505-4c4e-89ec-903a5a1dbed2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:57.079542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ba9f18-d505-4c4e-89ec-903a5a1dbed2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:57.081182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-41ba9f18-d505-4c4e-89ec-903a5a1dbed2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:57.151526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:57.190252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba9f18-d505-4c4e-89ec-903a5a1dbed2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:56:57.191602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ba9f18-d505-4c4e-89ec-903a5a1dbed2 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:56:57.210306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 667/1323] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:56:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:56:59.280240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3c24e-21a5-49b5-a922-7ebec0e96cf2 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:59.280742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3c24e-21a5-49b5-a922-7ebec0e96cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:59.280834 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab3c24e-21a5-49b5-a922-7ebec0e96cf2 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:59.282223 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 657/1324] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:56:59.287349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085e83a-dd63-45a9-b9f7-1ca0b84c3d97 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:59.287792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085e83a-dd63-45a9-b9f7-1ca0b84c3d97 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:59.287912 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e085e83a-dd63-45a9-b9f7-1ca0b84c3d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:59.351259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e085e83a-dd63-45a9-b9f7-1ca0b84c3d97 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:56:59.352441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e085e83a-dd63-45a9-b9f7-1ca0b84c3d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:59.447042 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 668/1325] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:56:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:56:59.495818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb46427-9618-40a2-883b-231ee526b0d4 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:59.496428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb46427-9618-40a2-883b-231ee526b0d4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:59.496656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb46427-9618-40a2-883b-231ee526b0d4 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:59.498091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 658/1326] 10.222.0.13 () {54 vars in 964 bytes} [Sun Jun 26 02:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:56:59.504153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcb2d8-1358-43d1-bc71-2f7732b7e8f1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:59.504448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcb2d8-1358-43d1-bc71-2f7732b7e8f1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:59.504727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcb2d8-1358-43d1-bc71-2f7732b7e8f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:59.534731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:59.579692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5cbcb2d8-1358-43d1-bc71-2f7732b7e8f1 None None] MFA Rules not processed for user `8ff708d6758d4f7b913e763824f3b791`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:56:59.580907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5cbcb2d8-1358-43d1-bc71-2f7732b7e8f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:59.660492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 669/1327] 10.222.0.13 () {58 vars in 1061 bytes} [Sun Jun 26 02:56:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:56:59.670205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb46427-9618-40a2-883b-231ee526b0d4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:56:59.671739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5bb46427-9618-40a2-883b-231ee526b0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:59.777748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb46427-9618-40a2-883b-231ee526b0d4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:56:59.859158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6fccd-27a0-4c14-bedc-0c18d806cc3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:56:59.859807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6fccd-27a0-4c14-bedc-0c18d806cc3e service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:56:59.860088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6fccd-27a0-4c14-bedc-0c18d806cc3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:56:59.860359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6fccd-27a0-4c14-bedc-0c18d806cc3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:56:59.861990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-52b6fccd-27a0-4c14-bedc-0c18d806cc3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:56:59.880642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:56:59.971283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6fccd-27a0-4c14-bedc-0c18d806cc3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:56:59.972720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b6fccd-27a0-4c14-bedc-0c18d806cc3e service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:56:59.991664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 659/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:56:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:02.285607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569d05f-a895-46f3-80be-44d63b67a59e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:02.286168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569d05f-a895-46f3-80be-44d63b67a59e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:02.286263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569d05f-a895-46f3-80be-44d63b67a59e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:02.287651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 670/1329] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:02.293252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b437f8-a5be-44af-9d2c-526c37c2e300 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:02.293662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b437f8-a5be-44af-9d2c-526c37c2e300 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:02.293823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b437f8-a5be-44af-9d2c-526c37c2e300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:02.358525 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-82b437f8-a5be-44af-9d2c-526c37c2e300 None None] MFA Rules not processed for user `19c972280f644c03b032e16fcb039658`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:02.359736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-82b437f8-a5be-44af-9d2c-526c37c2e300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:02.447984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:02.481897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 660/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:02.520548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569d05f-a895-46f3-80be-44d63b67a59e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:02.522113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e569d05f-a895-46f3-80be-44d63b67a59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:02.624152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569d05f-a895-46f3-80be-44d63b67a59e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:02.644212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:02.697637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598dc2f-e513-44d4-b575-aa3f2db9ff3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:02.698243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598dc2f-e513-44d4-b575-aa3f2db9ff3e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:02.698523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598dc2f-e513-44d4-b575-aa3f2db9ff3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:02.698792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598dc2f-e513-44d4-b575-aa3f2db9ff3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:02.700412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1598dc2f-e513-44d4-b575-aa3f2db9ff3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:02.808615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598dc2f-e513-44d4-b575-aa3f2db9ff3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:02.810065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1598dc2f-e513-44d4-b575-aa3f2db9ff3e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:02.827708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 671/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:02 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:05.103000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afbe28-9f6b-40d8-9800-8a5841927c36 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:05.103503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afbe28-9f6b-40d8-9800-8a5841927c36 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:05.103503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98afbe28-9f6b-40d8-9800-8a5841927c36 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:05.104897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 661/1332] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:05.110004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e40cfe-bdb0-43b4-bc39-84b78b7022fa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:05.110404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e40cfe-bdb0-43b4-bc39-84b78b7022fa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:05.110586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e40cfe-bdb0-43b4-bc39-84b78b7022fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:05.171989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:05.174711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-66e40cfe-bdb0-43b4-bc39-84b78b7022fa None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:05.176574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-66e40cfe-bdb0-43b4-bc39-84b78b7022fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:05.271463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 672/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 02:57:05.306164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afbe28-9f6b-40d8-9800-8a5841927c36 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:05.307732 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-98afbe28-9f6b-40d8-9800-8a5841927c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:05.410113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98afbe28-9f6b-40d8-9800-8a5841927c36 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:05.481526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff11cd16-bbd9-4406-ae7e-747566d5a793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:05.482197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11cd16-bbd9-4406-ae7e-747566d5a793 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:05.482481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11cd16-bbd9-4406-ae7e-747566d5a793 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:05.482755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff11cd16-bbd9-4406-ae7e-747566d5a793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:05.484367 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ff11cd16-bbd9-4406-ae7e-747566d5a793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:05.511451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:05.591790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11cd16-bbd9-4406-ae7e-747566d5a793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:05.593217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff11cd16-bbd9-4406-ae7e-747566d5a793 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:05.610815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 662/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:07.658562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546223c7-5401-4f2c-a2af-f535f503adbf None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:07.659127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546223c7-5401-4f2c-a2af-f535f503adbf None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:07.659127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546223c7-5401-4f2c-a2af-f535f503adbf None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:07.660492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 673/1335] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:07.666387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed6ad-ff29-494e-85b0-cd18a27a000d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:07.666819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed6ad-ff29-494e-85b0-cd18a27a000d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:07.666949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6ed6ad-ff29-494e-85b0-cd18a27a000d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:07.730396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1f6ed6ad-ff29-494e-85b0-cd18a27a000d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:07.731586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1f6ed6ad-ff29-494e-85b0-cd18a27a000d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:07.803256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:07.824906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 663/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:07.864691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546223c7-5401-4f2c-a2af-f535f503adbf None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:07.866336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-546223c7-5401-4f2c-a2af-f535f503adbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:07.966452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546223c7-5401-4f2c-a2af-f535f503adbf None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:07.995060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:08.039549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6217195-fec5-482e-88c2-7b9dd5ede457 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:08.040185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6217195-fec5-482e-88c2-7b9dd5ede457 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:08.040463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6217195-fec5-482e-88c2-7b9dd5ede457 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:08.040734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6217195-fec5-482e-88c2-7b9dd5ede457 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:08.042364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a6217195-fec5-482e-88c2-7b9dd5ede457 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:08.147494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6217195-fec5-482e-88c2-7b9dd5ede457 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:08.148916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6217195-fec5-482e-88c2-7b9dd5ede457 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:08.167156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 674/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:10.580141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9c092-fad8-4f58-bb17-ba826401b534 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:10.580673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9c092-fad8-4f58-bb17-ba826401b534 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:10.580798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed9c092-fad8-4f58-bb17-ba826401b534 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:10.582125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 664/1338] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:10.587324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21f757-96e1-4f73-ae31-090e5b2869b5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:10.587769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21f757-96e1-4f73-ae31-090e5b2869b5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:10.587874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21f757-96e1-4f73-ae31-090e5b2869b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:10.652983 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4d21f757-96e1-4f73-ae31-090e5b2869b5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:10.654200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4d21f757-96e1-4f73-ae31-090e5b2869b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:10.662809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:10.748929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 675/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:10.784719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9c092-fad8-4f58-bb17-ba826401b534 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:10.786912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2ed9c092-fad8-4f58-bb17-ba826401b534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:10.890489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed9c092-fad8-4f58-bb17-ba826401b534 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:10.969423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3034c88-04d8-4a17-accc-76d6c88f27a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:10.970092 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3034c88-04d8-4a17-accc-76d6c88f27a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:10.970379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3034c88-04d8-4a17-accc-76d6c88f27a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:10.970654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3034c88-04d8-4a17-accc-76d6c88f27a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:10.972296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d3034c88-04d8-4a17-accc-76d6c88f27a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:11.008897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:11.080670 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3034c88-04d8-4a17-accc-76d6c88f27a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:11.082124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3034c88-04d8-4a17-accc-76d6c88f27a4 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:11.099971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 665/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:15.016571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f129f-aa2e-422b-a6e9-6b6010a44994 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:15.017121 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f129f-aa2e-422b-a6e9-6b6010a44994 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:15.017243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56f129f-aa2e-422b-a6e9-6b6010a44994 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:15.018637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 676/1341] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:15.023988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4d84-7f76-4ef8-8907-207f43de13e1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:15.024388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4d84-7f76-4ef8-8907-207f43de13e1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:15.024555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d4d84-7f76-4ef8-8907-207f43de13e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:15.095592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e85d4d84-7f76-4ef8-8907-207f43de13e1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:15.096801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e85d4d84-7f76-4ef8-8907-207f43de13e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:15.189428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:15.202859 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 666/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:15.242464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f129f-aa2e-422b-a6e9-6b6010a44994 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:15.244621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c56f129f-aa2e-422b-a6e9-6b6010a44994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:15.341330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56f129f-aa2e-422b-a6e9-6b6010a44994 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:15.344911 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4593cbf-2402-4b80-a761-99806846ce30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:15.345556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593cbf-2402-4b80-a761-99806846ce30 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:15.345843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593cbf-2402-4b80-a761-99806846ce30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:15.346160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4593cbf-2402-4b80-a761-99806846ce30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:15.355715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4593cbf-2402-4b80-a761-99806846ce30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:15.357190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4593cbf-2402-4b80-a761-99806846ce30 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:15.367285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 677/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:18.048700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90120786-eca8-457d-9a84-6a2556fe5f7e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:18.049239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90120786-eca8-457d-9a84-6a2556fe5f7e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:18.049239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90120786-eca8-457d-9a84-6a2556fe5f7e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:18.050715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 667/1344] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:18.056623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102db06-c3bd-496a-809e-00eb61b10b5e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:18.057076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102db06-c3bd-496a-809e-00eb61b10b5e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:18.057354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2102db06-c3bd-496a-809e-00eb61b10b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:18.084558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2102db06-c3bd-496a-809e-00eb61b10b5e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:18.085770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2102db06-c3bd-496a-809e-00eb61b10b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:18.114885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:18.168804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 678/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:18.206342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90120786-eca8-457d-9a84-6a2556fe5f7e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:18.217840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90120786-eca8-457d-9a84-6a2556fe5f7e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:18.221375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f496b818-54dc-490d-89b3-6dc28a064c1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:18.222143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496b818-54dc-490d-89b3-6dc28a064c1e service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:18.222427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496b818-54dc-490d-89b3-6dc28a064c1e service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:18.222701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f496b818-54dc-490d-89b3-6dc28a064c1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:18.233330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496b818-54dc-490d-89b3-6dc28a064c1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:18.234809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f496b818-54dc-490d-89b3-6dc28a064c1e service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:18.246310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 668/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:22.219100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e18a-6397-4a53-99d1-2339d6f4e80a None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:22.219617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e18a-6397-4a53-99d1-2339d6f4e80a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:22.219740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287e18a-6397-4a53-99d1-2339d6f4e80a None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:22.221101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 679/1347] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:22.226925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3d9d7-ba04-4ab9-b901-9ee79e7699d5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:22.227371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3d9d7-ba04-4ab9-b901-9ee79e7699d5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:22.227495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3d9d7-ba04-4ab9-b901-9ee79e7699d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:22.245883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-32e3d9d7-ba04-4ab9-b901-9ee79e7699d5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:22.247128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-32e3d9d7-ba04-4ab9-b901-9ee79e7699d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:22.255316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 669/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:22.293591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287e18a-6397-4a53-99d1-2339d6f4e80a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:22.295371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2287e18a-6397-4a53-99d1-2339d6f4e80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:22.311885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287e18a-6397-4a53-99d1-2339d6f4e80a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:22.315408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b32d264-e936-4dca-8caf-15e93abb8e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:22.316057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32d264-e936-4dca-8caf-15e93abb8e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:22.316337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32d264-e936-4dca-8caf-15e93abb8e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:22.316652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b32d264-e936-4dca-8caf-15e93abb8e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:22.326125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32d264-e936-4dca-8caf-15e93abb8e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:22.327549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b32d264-e936-4dca-8caf-15e93abb8e5b service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:22.337027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 680/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 02:57:27.533197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf47fe-3da0-4f77-b965-702265c5f6a9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:27.533609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf47fe-3da0-4f77-b965-702265c5f6a9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:27.536972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf47fe-3da0-4f77-b965-702265c5f6a9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:27.536972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 670/1350] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 02:57:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:27.542236 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fb3e6-61d4-4254-9f08-ce72ab8a58e9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:27.543072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fb3e6-61d4-4254-9f08-ce72ab8a58e9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:27.543558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fb3e6-61d4-4254-9f08-ce72ab8a58e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:27.668957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a73fb3e6-61d4-4254-9f08-ce72ab8a58e9 None None] MFA Rules not processed for user `19ce6f259c22441486bfc04811a271b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:27.670871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a73fb3e6-61d4-4254-9f08-ce72ab8a58e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:27.753946 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 681/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 02:57:27 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:32.024697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b765f-0b8b-4c76-886c-9c360621c364 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:32.025777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b765f-0b8b-4c76-886c-9c360621c364 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:32.026180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2b765f-0b8b-4c76-886c-9c360621c364 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:32.027817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 671/1352] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 02:57:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:32.033178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742542d-2456-406e-a342-0e4a5054bf15 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:32.033617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742542d-2456-406e-a342-0e4a5054bf15 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:32.033906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2742542d-2456-406e-a342-0e4a5054bf15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:32.054494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2742542d-2456-406e-a342-0e4a5054bf15 None None] MFA Rules not processed for user `19ce6f259c22441486bfc04811a271b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:32.055786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2742542d-2456-406e-a342-0e4a5054bf15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:32.064665 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 682/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 02:57:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:32.240264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=13 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:33.519045 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3ad10-3a31-4d9e-9c12-3840e1825e82 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:33.519668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3ad10-3a31-4d9e-9c12-3840e1825e82 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:33.519668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a3ad10-3a31-4d9e-9c12-3840e1825e82 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:33.521053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 672/1354] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 02:57:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:33.526216 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcb2a3-b444-48fa-943a-ea4584d7e068 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:33.526656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcb2a3-b444-48fa-943a-ea4584d7e068 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:33.526780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dcb2a3-b444-48fa-943a-ea4584d7e068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:33.545712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-53dcb2a3-b444-48fa-943a-ea4584d7e068 None None] MFA Rules not processed for user `19ce6f259c22441486bfc04811a271b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:33.546917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-53dcb2a3-b444-48fa-943a-ea4584d7e068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:33.556269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 683/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 02:57:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 02:57:37.150575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d86f6-b9df-45ac-8550-02a558d6ffa3 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:37.151555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d86f6-b9df-45ac-8550-02a558d6ffa3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:37.152421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d86f6-b9df-45ac-8550-02a558d6ffa3 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:37.153921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 673/1356] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:37.159418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31b53f-135c-47f5-b63e-557d34eecc0c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:37.159836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31b53f-135c-47f5-b63e-557d34eecc0c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:37.160268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c31b53f-135c-47f5-b63e-557d34eecc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:37.179318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1c31b53f-135c-47f5-b63e-557d34eecc0c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:37.180512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1c31b53f-135c-47f5-b63e-557d34eecc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:37.188700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 684/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:37.247753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c8cda-e5c1-42a7-b05e-9e9206ce754a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:37.249100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c8cda-e5c1-42a7-b05e-9e9206ce754a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:37.249953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c8cda-e5c1-42a7-b05e-9e9206ce754a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:37.251757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 674/1358] 10.222.0.13 () {54 vars in 961 bytes} [Sun Jun 26 02:57:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:37.261037 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caac12-b018-40df-bb18-bb6bf1e39d1f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:37.262296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caac12-b018-40df-bb18-bb6bf1e39d1f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:37.263106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3caac12-b018-40df-bb18-bb6bf1e39d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:37.307727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:37.316614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e3caac12-b018-40df-bb18-bb6bf1e39d1f None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:37.317830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e3caac12-b018-40df-bb18-bb6bf1e39d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:37.404968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 685/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Jun 26 02:57:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 02:57:37.413501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c8cda-e5c1-42a7-b05e-9e9206ce754a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:37.428278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c8cda-e5c1-42a7-b05e-9e9206ce754a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:37.433688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a9a9a-d7d7-4fe2-bf50-7918e13cdd3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:37.434360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a9a9a-d7d7-4fe2-bf50-7918e13cdd3f service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:37.434646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a9a9a-d7d7-4fe2-bf50-7918e13cdd3f service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:37.434920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a9a9a-d7d7-4fe2-bf50-7918e13cdd3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:37.443581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a9a9a-d7d7-4fe2-bf50-7918e13cdd3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:37.445155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419a9a9a-d7d7-4fe2-bf50-7918e13cdd3f service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:37.454898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 675/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 02:57:39.684145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306744f4-b997-4eb3-9b46-064ab50d75b5 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:39.684672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306744f4-b997-4eb3-9b46-064ab50d75b5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:39.684790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306744f4-b997-4eb3-9b46-064ab50d75b5 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:39.686303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 686/1361] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:39.691914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0c95-782b-45c0-bb78-9af56573351b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:39.692327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0c95-782b-45c0-bb78-9af56573351b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:39.692474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48a0c95-782b-45c0-bb78-9af56573351b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:39.710947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c48a0c95-782b-45c0-bb78-9af56573351b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:39.712167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c48a0c95-782b-45c0-bb78-9af56573351b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:39.720070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 676/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:39.764708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306744f4-b997-4eb3-9b46-064ab50d75b5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:39.779998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306744f4-b997-4eb3-9b46-064ab50d75b5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:39.785576 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47bdac-d97d-4177-9c08-7d520dbdeba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:39.786356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47bdac-d97d-4177-9c08-7d520dbdeba1 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:39.786766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47bdac-d97d-4177-9c08-7d520dbdeba1 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:39.787156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47bdac-d97d-4177-9c08-7d520dbdeba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:39.795590 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47bdac-d97d-4177-9c08-7d520dbdeba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:39.797288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47bdac-d97d-4177-9c08-7d520dbdeba1 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:39.807042 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 687/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:42.182203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc737b-5ce0-4e6e-8b26-09c42d7c10da None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:42.182676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc737b-5ce0-4e6e-8b26-09c42d7c10da None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:42.182889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bc737b-5ce0-4e6e-8b26-09c42d7c10da None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:42.184178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 677/1364] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:42.189593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74244c1-9049-4f20-93de-903df79dbc9a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:42.190103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74244c1-9049-4f20-93de-903df79dbc9a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:42.190213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74244c1-9049-4f20-93de-903df79dbc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:42.210553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d74244c1-9049-4f20-93de-903df79dbc9a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:42.211762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d74244c1-9049-4f20-93de-903df79dbc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:42.220081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 688/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 02:57:42.266669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b817610-ab28-4090-9fb7-b6676a7041ef None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:42.267178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b817610-ab28-4090-9fb7-b6676a7041ef None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:42.267465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b817610-ab28-4090-9fb7-b6676a7041ef None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:42.268875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 678/1366] 10.222.0.13 () {54 vars in 961 bytes} [Sun Jun 26 02:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:42.274744 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de90e-3419-4237-90f6-0f299e34ca41 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:42.275030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de90e-3419-4237-90f6-0f299e34ca41 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:42.275304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09de90e-3419-4237-90f6-0f299e34ca41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:42.295126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d09de90e-3419-4237-90f6-0f299e34ca41 None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:42.296381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d09de90e-3419-4237-90f6-0f299e34ca41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:42.312007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 689/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Jun 26 02:57:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:42.322512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b817610-ab28-4090-9fb7-b6676a7041ef None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:42.338179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b817610-ab28-4090-9fb7-b6676a7041ef None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:42.343800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe5a26-10cb-42cb-9fc6-65bfd2202530 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:42.344452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5a26-10cb-42cb-9fc6-65bfd2202530 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:42.344743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5a26-10cb-42cb-9fc6-65bfd2202530 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:42.345029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe5a26-10cb-42cb-9fc6-65bfd2202530 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:42.355235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe5a26-10cb-42cb-9fc6-65bfd2202530 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:42.356818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe5a26-10cb-42cb-9fc6-65bfd2202530 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:42.366590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 679/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:44.457634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ede3b-abf1-4e1b-8cfe-e37279a6e2c7 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:44.458201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ede3b-abf1-4e1b-8cfe-e37279a6e2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:44.458201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ede3b-abf1-4e1b-8cfe-e37279a6e2c7 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:44.459821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 690/1369] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:44.464704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92b9e-fece-4286-af33-a20c7c8d584f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:44.465137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92b9e-fece-4286-af33-a20c7c8d584f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:44.465436 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa92b9e-fece-4286-af33-a20c7c8d584f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:44.483222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6fa92b9e-fece-4286-af33-a20c7c8d584f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:44.484703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6fa92b9e-fece-4286-af33-a20c7c8d584f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:44.493048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 680/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:44.536896 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ede3b-abf1-4e1b-8cfe-e37279a6e2c7 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:44.551668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ede3b-abf1-4e1b-8cfe-e37279a6e2c7 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:44.557237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336a41a2-51be-4adb-aebc-34a2df55bc48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:44.557879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a41a2-51be-4adb-aebc-34a2df55bc48 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:44.558200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a41a2-51be-4adb-aebc-34a2df55bc48 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:44.558484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336a41a2-51be-4adb-aebc-34a2df55bc48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:44.567413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a41a2-51be-4adb-aebc-34a2df55bc48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:44.568984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336a41a2-51be-4adb-aebc-34a2df55bc48 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:44.578741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 691/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:45.310683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff2ef-4e03-42f1-80ae-97c7b43d8ad1 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:45.311055 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff2ef-4e03-42f1-80ae-97c7b43d8ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:45.311338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff2ef-4e03-42f1-80ae-97c7b43d8ad1 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:45.312784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 681/1372] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:45.318145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3b7e5-b4af-49b9-914a-285648b74d0f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:45.318429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3b7e5-b4af-49b9-914a-285648b74d0f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:45.318711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3b7e5-b4af-49b9-914a-285648b74d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:45.336919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6ca3b7e5-b4af-49b9-914a-285648b74d0f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:45.338128 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6ca3b7e5-b4af-49b9-914a-285648b74d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:45.345967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 692/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:45.685487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c247e0-6a3a-4c58-a3a2-a52cc5c52087 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:45.685487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c247e0-6a3a-4c58-a3a2-a52cc5c52087 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:45.685487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c247e0-6a3a-4c58-a3a2-a52cc5c52087 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:45.685487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 682/1374] 10.222.0.13 () {54 vars in 963 bytes} [Sun Jun 26 02:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:45.689190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed1b7-8ab4-4c44-bca1-30810e3da5d0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:45.689190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed1b7-8ab4-4c44-bca1-30810e3da5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:45.689190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed1b7-8ab4-4c44-bca1-30810e3da5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:45.743889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-997ed1b7-8ab4-4c44-bca1-30810e3da5d0 None None] MFA Rules not processed for user `5e65f36f772b4220a909f8fda83ba42e`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:45.745089 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-997ed1b7-8ab4-4c44-bca1-30810e3da5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:45.807602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 693/1375] 10.222.0.13 () {58 vars in 1060 bytes} [Sun Jun 26 02:57:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:45.815287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c247e0-6a3a-4c58-a3a2-a52cc5c52087 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:45.826382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c247e0-6a3a-4c58-a3a2-a52cc5c52087 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:45.829764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f5408-912a-49a6-b2a0-2387fd6d488e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:45.830422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f5408-912a-49a6-b2a0-2387fd6d488e service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:45.830704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f5408-912a-49a6-b2a0-2387fd6d488e service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:45.830976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f5408-912a-49a6-b2a0-2387fd6d488e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:45.840588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f5408-912a-49a6-b2a0-2387fd6d488e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:45.842019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97f5408-912a-49a6-b2a0-2387fd6d488e service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:45.856531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 683/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:46.662216 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1533253-a4b4-492b-85eb-d23d9a27e99f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:46.662569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1533253-a4b4-492b-85eb-d23d9a27e99f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:46.662940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1533253-a4b4-492b-85eb-d23d9a27e99f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:46.664419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 694/1377] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:46.670399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09484abf-bff0-4981-abba-2cf216aba691 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:46.670909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09484abf-bff0-4981-abba-2cf216aba691 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:46.671332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09484abf-bff0-4981-abba-2cf216aba691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:46.692219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-09484abf-bff0-4981-abba-2cf216aba691 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:46.694339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-09484abf-bff0-4981-abba-2cf216aba691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:46.710766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 684/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:46.747798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1533253-a4b4-492b-85eb-d23d9a27e99f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:46.766396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1533253-a4b4-492b-85eb-d23d9a27e99f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:46.773778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a621a-a388-4657-b518-68c2d97d3a52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:46.774686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a621a-a388-4657-b518-68c2d97d3a52 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:46.775147 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a621a-a388-4657-b518-68c2d97d3a52 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:46.775657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a621a-a388-4657-b518-68c2d97d3a52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:46.786272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a621a-a388-4657-b518-68c2d97d3a52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:46.787909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7a621a-a388-4657-b518-68c2d97d3a52 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:46.798686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:57:46.804586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 695/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:48.868914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad099ff-5382-4de3-973a-f9f587e34ffd None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:48.869806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad099ff-5382-4de3-973a-f9f587e34ffd None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:48.870284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad099ff-5382-4de3-973a-f9f587e34ffd None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:48.872845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 685/1380] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:48.877845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483c08f-695f-4bde-810c-435b42c40914 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:48.878382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483c08f-695f-4bde-810c-435b42c40914 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:48.878490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4483c08f-695f-4bde-810c-435b42c40914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:48.896583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4483c08f-695f-4bde-810c-435b42c40914 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:48.897784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4483c08f-695f-4bde-810c-435b42c40914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:48.905812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 696/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:48.950807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad099ff-5382-4de3-973a-f9f587e34ffd None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:48.966365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad099ff-5382-4de3-973a-f9f587e34ffd None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:48.971735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9d38e2-a10e-4d93-bdfc-eafc2ee369bf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:48.973002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d38e2-a10e-4d93-bdfc-eafc2ee369bf service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:48.973272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d38e2-a10e-4d93-bdfc-eafc2ee369bf service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:48.973549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9d38e2-a10e-4d93-bdfc-eafc2ee369bf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:48.981710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d38e2-a10e-4d93-bdfc-eafc2ee369bf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:48.983378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9d38e2-a10e-4d93-bdfc-eafc2ee369bf service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:48.993021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 686/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:51.082890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ef1dc-d4c5-42b7-88e4-029dd156898f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:51.083349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ef1dc-d4c5-42b7-88e4-029dd156898f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:51.083431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543ef1dc-d4c5-42b7-88e4-029dd156898f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:51.084839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 697/1383] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:51.090076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98353a6d-b1c4-485a-ade3-9510ae31cb19 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:51.090480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98353a6d-b1c4-485a-ade3-9510ae31cb19 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:51.090646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98353a6d-b1c4-485a-ade3-9510ae31cb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:51.109427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-98353a6d-b1c4-485a-ade3-9510ae31cb19 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:51.110857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-98353a6d-b1c4-485a-ade3-9510ae31cb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:51.119292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 687/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:51.161098 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ef1dc-d4c5-42b7-88e4-029dd156898f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:51.175942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543ef1dc-d4c5-42b7-88e4-029dd156898f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:51.181438 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f27230-35e6-4f5a-8621-0da5f96a4c4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:51.182104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f27230-35e6-4f5a-8621-0da5f96a4c4b service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:51.182388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f27230-35e6-4f5a-8621-0da5f96a4c4b service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:51.182659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f27230-35e6-4f5a-8621-0da5f96a4c4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:51.192619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f27230-35e6-4f5a-8621-0da5f96a4c4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:51.194256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f27230-35e6-4f5a-8621-0da5f96a4c4b service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:51.209479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 698/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:53.250532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57888d-169a-4c0f-95e0-8f2d53bf3d25 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:53.251338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57888d-169a-4c0f-95e0-8f2d53bf3d25 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:53.251338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57888d-169a-4c0f-95e0-8f2d53bf3d25 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:53.258231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ff758-6532-4f86-9544-1cde7ead7f00 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:53.258231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ff758-6532-4f86-9544-1cde7ead7f00 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:53.258984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ff758-6532-4f86-9544-1cde7ead7f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:53.260646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 688/1386] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:53 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:53.277375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d61ff758-6532-4f86-9544-1cde7ead7f00 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:53.278705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d61ff758-6532-4f86-9544-1cde7ead7f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:53.288305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 699/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 02:57:53.333252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57888d-169a-4c0f-95e0-8f2d53bf3d25 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:53.349367 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c57888d-169a-4c0f-95e0-8f2d53bf3d25 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:53.354770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67fc89-9827-4d39-98eb-0b8516f7baf8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:53.355397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67fc89-9827-4d39-98eb-0b8516f7baf8 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:53.355665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67fc89-9827-4d39-98eb-0b8516f7baf8 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:53.355946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67fc89-9827-4d39-98eb-0b8516f7baf8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:53.364960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67fc89-9827-4d39-98eb-0b8516f7baf8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:53.366558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67fc89-9827-4d39-98eb-0b8516f7baf8 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:53.376510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 689/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:55.485467 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494247c-0c51-4dd1-8dc5-7d479d4ce17c None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:55.486489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494247c-0c51-4dd1-8dc5-7d479d4ce17c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:55.486489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f494247c-0c51-4dd1-8dc5-7d479d4ce17c None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:55.487714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 700/1389] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:55.494904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ae798-4cb3-43d6-a2c2-9f2ea0a7c1bc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:55.495355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ae798-4cb3-43d6-a2c2-9f2ea0a7c1bc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:55.495568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ae798-4cb3-43d6-a2c2-9f2ea0a7c1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:55.517792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-704ae798-4cb3-43d6-a2c2-9f2ea0a7c1bc None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:55.519734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-704ae798-4cb3-43d6-a2c2-9f2ea0a7c1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:55.528540 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 690/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 02:57:55.572891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494247c-0c51-4dd1-8dc5-7d479d4ce17c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:55.589566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f494247c-0c51-4dd1-8dc5-7d479d4ce17c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:55.596487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1746656f-01a3-4203-8431-78dec8bc8c79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:55.597285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746656f-01a3-4203-8431-78dec8bc8c79 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:55.597640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746656f-01a3-4203-8431-78dec8bc8c79 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:55.597995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1746656f-01a3-4203-8431-78dec8bc8c79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:55.608693 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746656f-01a3-4203-8431-78dec8bc8c79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:55.610786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1746656f-01a3-4203-8431-78dec8bc8c79 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:55.623358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 701/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:57.742791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b7ce-88a8-4504-8dd3-998c45e5072b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:57.743516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b7ce-88a8-4504-8dd3-998c45e5072b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:57.743516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d0b7ce-88a8-4504-8dd3-998c45e5072b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:57.744777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 691/1392] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:57.750363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9ed03-ede5-4ac6-8cbb-f425f19ef66a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:57.750787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9ed03-ede5-4ac6-8cbb-f425f19ef66a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:57.750924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9ed03-ede5-4ac6-8cbb-f425f19ef66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:57.768967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-24f9ed03-ede5-4ac6-8cbb-f425f19ef66a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:57.770157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-24f9ed03-ede5-4ac6-8cbb-f425f19ef66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:57.778250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 702/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:57:57.824960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0b7ce-88a8-4504-8dd3-998c45e5072b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:57:57.849462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d0b7ce-88a8-4504-8dd3-998c45e5072b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:57:57.858988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977cc91-5c8c-41b1-a948-0cea7c0db41c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:57:57.860155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977cc91-5c8c-41b1-a948-0cea7c0db41c service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:57.860655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977cc91-5c8c-41b1-a948-0cea7c0db41c service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:57.861168 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977cc91-5c8c-41b1-a948-0cea7c0db41c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:57.876978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977cc91-5c8c-41b1-a948-0cea7c0db41c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:57:57.879917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977cc91-5c8c-41b1-a948-0cea7c0db41c service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:57:57.897251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 692/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:57:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:57:59.955267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fc431-261a-489f-bac1-820074d92f56 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:59.955840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fc431-261a-489f-bac1-820074d92f56 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:59.955840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723fc431-261a-489f-bac1-820074d92f56 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:59.957247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 703/1395] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:57:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:57:59.962449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e3e24-1e70-4922-9331-c788614bceea None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:57:59.962872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e3e24-1e70-4922-9331-c788614bceea None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:57:59.963005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e3e24-1e70-4922-9331-c788614bceea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:57:59.980614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e84e3e24-1e70-4922-9331-c788614bceea None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:57:59.981794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e84e3e24-1e70-4922-9331-c788614bceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:57:59.990683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 693/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 02:58:00.033529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fc431-261a-489f-bac1-820074d92f56 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:00.048193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723fc431-261a-489f-bac1-820074d92f56 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:00.054095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaecdf3-7dfc-4f35-a918-43efdb5c1fa3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:00.054723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaecdf3-7dfc-4f35-a918-43efdb5c1fa3 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:00.055006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaecdf3-7dfc-4f35-a918-43efdb5c1fa3 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:00.055277 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaecdf3-7dfc-4f35-a918-43efdb5c1fa3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:00.063266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaecdf3-7dfc-4f35-a918-43efdb5c1fa3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:00.064829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaecdf3-7dfc-4f35-a918-43efdb5c1fa3 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:00.073926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 704/1397] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:01.151637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda076f1-fdb1-449a-a8d8-f780ecfcf32b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:01.152198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda076f1-fdb1-449a-a8d8-f780ecfcf32b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:01.152198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda076f1-fdb1-449a-a8d8-f780ecfcf32b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:01.153712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 694/1398] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:01.158865 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b5c5-9362-4cc5-9b71-174ebcb6b658 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:01.159342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b5c5-9362-4cc5-9b71-174ebcb6b658 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:01.159458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7b5c5-9362-4cc5-9b71-174ebcb6b658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:01.177415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b6a7b5c5-9362-4cc5-9b71-174ebcb6b658 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:01.178652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b6a7b5c5-9362-4cc5-9b71-174ebcb6b658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:01.188484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 705/1399] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:58:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:01.249634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1359650-223c-472e-9af2-7f35c23dc5ca None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:01.249918 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1359650-223c-472e-9af2-7f35c23dc5ca None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:01.250257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1359650-223c-472e-9af2-7f35c23dc5ca None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:01.251657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 695/1400] 10.222.0.13 () {54 vars in 962 bytes} [Sun Jun 26 02:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:01.256943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f705f8-65ac-4cf0-9c3e-1d963b767fb3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:01.257237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f705f8-65ac-4cf0-9c3e-1d963b767fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:01.257515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f705f8-65ac-4cf0-9c3e-1d963b767fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:01.304078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-51f705f8-65ac-4cf0-9c3e-1d963b767fb3 None None] MFA Rules not processed for user `0dc5b601421a41d1a30a10e86967a920`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:01.305255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-51f705f8-65ac-4cf0-9c3e-1d963b767fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:01.364428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 706/1401] 10.222.0.13 () {58 vars in 1059 bytes} [Sun Jun 26 02:58:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:01.372124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1359650-223c-472e-9af2-7f35c23dc5ca None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:01.382676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1359650-223c-472e-9af2-7f35c23dc5ca None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:01.386100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09d826-80f9-4cf0-8d57-e2f76f9d79f9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0dc5b601421a41d1a30a10e86967a920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:01.386724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09d826-80f9-4cf0-8d57-e2f76f9d79f9 service glance] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:01.387006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09d826-80f9-4cf0-8d57-e2f76f9d79f9 service glance] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:01.387284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09d826-80f9-4cf0-8d57-e2f76f9d79f9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:01.397304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09d826-80f9-4cf0-8d57-e2f76f9d79f9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:01.398747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09d826-80f9-4cf0-8d57-e2f76f9d79f9 service glance] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:01.408293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 696/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:02.115312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34446f8-8f16-4356-823d-6051f9b3abf5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:02.115604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34446f8-8f16-4356-823d-6051f9b3abf5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:02.115883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34446f8-8f16-4356-823d-6051f9b3abf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:02.121255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8043a-811f-431c-9094-0f316398f276 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:02.121632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8043a-811f-431c-9094-0f316398f276 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:02.121985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8043a-811f-431c-9094-0f316398f276 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:02.123494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 697/1403] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:58:02.129366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb6286-62f9-49d2-8161-5b5138b46b17 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:02.129731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb6286-62f9-49d2-8161-5b5138b46b17 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:02.130108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceb6286-62f9-49d2-8161-5b5138b46b17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:02.148529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-aceb6286-62f9-49d2-8161-5b5138b46b17 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:02.149727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-aceb6286-62f9-49d2-8161-5b5138b46b17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:02.159012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 698/1404] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:58:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:02.167081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b34446f8-8f16-4356-823d-6051f9b3abf5 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:02.168261 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b34446f8-8f16-4356-823d-6051f9b3abf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:02.202940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb6286-62f9-49d2-8161-5b5138b46b17 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:02.210379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:58:02.216665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceb6286-62f9-49d2-8161-5b5138b46b17 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:02.222276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ea6d7-7623-4fec-b462-57c1a6dbc461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:02.222944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ea6d7-7623-4fec-b462-57c1a6dbc461 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:02.223232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ea6d7-7623-4fec-b462-57c1a6dbc461 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:02.223625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ea6d7-7623-4fec-b462-57c1a6dbc461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:02.231839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ea6d7-7623-4fec-b462-57c1a6dbc461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:02.233427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ea6d7-7623-4fec-b462-57c1a6dbc461 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:02.235183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 707/1405] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 02:58:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:02.244729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 699/1406] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 02:58:02.276900 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0db61e-ec4e-4d42-8d4b-cb1d2aedd6fb None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:02.277194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0db61e-ec4e-4d42-8d4b-cb1d2aedd6fb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:02.277550 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0db61e-ec4e-4d42-8d4b-cb1d2aedd6fb None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:02.279204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 708/1407] 10.222.0.13 () {54 vars in 942 bytes} [Sun Jun 26 02:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:02.285589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c5437-7d82-4d78-b47e-2103b355f067 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:02.285886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c5437-7d82-4d78-b47e-2103b355f067 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:02.286202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c5437-7d82-4d78-b47e-2103b355f067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:02.291677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=70 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:58:02.339908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0f1c5437-7d82-4d78-b47e-2103b355f067 None None] MFA Rules not processed for user `60f006c99809427496b026c17a663995`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:02.341177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0f1c5437-7d82-4d78-b47e-2103b355f067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:02.419224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 700/1408] 10.222.0.13 () {58 vars in 1039 bytes} [Sun Jun 26 02:58:02 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:02.429700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0db61e-ec4e-4d42-8d4b-cb1d2aedd6fb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:02.447304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0db61e-ec4e-4d42-8d4b-cb1d2aedd6fb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:02.452268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd944cb8-8be0-4b2b-87c6-780b34ee14c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:02.452921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd944cb8-8be0-4b2b-87c6-780b34ee14c5 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:02.453204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd944cb8-8be0-4b2b-87c6-780b34ee14c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:02.453482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd944cb8-8be0-4b2b-87c6-780b34ee14c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:02.462304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd944cb8-8be0-4b2b-87c6-780b34ee14c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:02.463895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd944cb8-8be0-4b2b-87c6-780b34ee14c5 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:02.472085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 709/1409] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 02:58:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 02:58:12.417242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=43 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:58:12.468696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=15 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:58:24.615388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61619c90-e371-4fa4-8aea-0304b10e5211 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:24.615912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61619c90-e371-4fa4-8aea-0304b10e5211 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:24.615912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61619c90-e371-4fa4-8aea-0304b10e5211 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:24.617321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 701/1410] 10.222.0.13 () {54 vars in 918 bytes} [Sun Jun 26 02:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:24.622465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eb639-d34a-47a7-95bb-94ee816a1832 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:24.622862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eb639-d34a-47a7-95bb-94ee816a1832 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:24.623062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924eb639-d34a-47a7-95bb-94ee816a1832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:24.641413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-924eb639-d34a-47a7-95bb-94ee816a1832 None None] MFA Rules not processed for user `19ce6f259c22441486bfc04811a271b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:24.642640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-924eb639-d34a-47a7-95bb-94ee816a1832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:24.651663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 710/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Sun Jun 26 02:58:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:24.665428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6635c-0db3-4519-b264-b54ccef67c8e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:24.665712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6635c-0db3-4519-b264-b54ccef67c8e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:24.665987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc6635c-0db3-4519-b264-b54ccef67c8e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:24.667386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 702/1412] 10.222.0.13 () {54 vars in 918 bytes} [Sun Jun 26 02:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:24.671816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69c1c8-a574-45c4-834f-48c80ed2ae97 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:24.672104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69c1c8-a574-45c4-834f-48c80ed2ae97 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:24.672380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a69c1c8-a574-45c4-834f-48c80ed2ae97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:24.689467 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6a69c1c8-a574-45c4-834f-48c80ed2ae97 None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:24.691174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6a69c1c8-a574-45c4-834f-48c80ed2ae97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:24.700351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 711/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Sun Jun 26 02:58:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:27.174435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57f71a-698b-4eb6-a659-65fe52cba18e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:27.175010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57f71a-698b-4eb6-a659-65fe52cba18e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:27.175010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57f71a-698b-4eb6-a659-65fe52cba18e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:27.176418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 703/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:27.181494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b129a-26ab-4a72-87f2-553294e4a104 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:27.181865 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b129a-26ab-4a72-87f2-553294e4a104 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:27.182025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b129a-26ab-4a72-87f2-553294e4a104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:27.199385 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d96b129a-26ab-4a72-87f2-553294e4a104 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:27.200559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d96b129a-26ab-4a72-87f2-553294e4a104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:27.208676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 712/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:27.252028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57f71a-698b-4eb6-a659-65fe52cba18e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:27.266707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57f71a-698b-4eb6-a659-65fe52cba18e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:27.272746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a948856-c11f-4de8-8706-b3f4876fab4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:27.273412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a948856-c11f-4de8-8706-b3f4876fab4c service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:27.273700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a948856-c11f-4de8-8706-b3f4876fab4c service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:27.273977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a948856-c11f-4de8-8706-b3f4876fab4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:27.282177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a948856-c11f-4de8-8706-b3f4876fab4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:27.283752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a948856-c11f-4de8-8706-b3f4876fab4c service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:27.293625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 704/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:28.191256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c42aee-2884-46c1-937a-17cf93a57ab0 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:28.191748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c42aee-2884-46c1-937a-17cf93a57ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:28.191873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c42aee-2884-46c1-937a-17cf93a57ab0 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:28.193197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 713/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 02:58:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:28.197962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd35e-6c6b-4e25-adfe-fb22bc57d2d3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:28.198382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd35e-6c6b-4e25-adfe-fb22bc57d2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:28.198558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cd35e-6c6b-4e25-adfe-fb22bc57d2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:28.215928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-645cd35e-6c6b-4e25-adfe-fb22bc57d2d3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:28.217106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-645cd35e-6c6b-4e25-adfe-fb22bc57d2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:28.225810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 705/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 02:58:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:28.373957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c42aee-2884-46c1-937a-17cf93a57ab0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:28.388109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c42aee-2884-46c1-937a-17cf93a57ab0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:28.393293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349e7c7-3763-48bb-84df-77be451351e7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:28.393947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349e7c7-3763-48bb-84df-77be451351e7 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:28.394256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349e7c7-3763-48bb-84df-77be451351e7 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:28.394530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349e7c7-3763-48bb-84df-77be451351e7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:28.402433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349e7c7-3763-48bb-84df-77be451351e7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:28.403926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5349e7c7-3763-48bb-84df-77be451351e7 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:28.413506 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 714/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:29.303411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50a49d-9a44-476e-90e9-92ccffc97960 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:29.303974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50a49d-9a44-476e-90e9-92ccffc97960 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:29.303974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee50a49d-9a44-476e-90e9-92ccffc97960 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:29.305506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 706/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 02:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:29.310630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67ac65-7854-46fd-87c5-66e11cb8f45a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:29.311077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67ac65-7854-46fd-87c5-66e11cb8f45a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:29.311174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae67ac65-7854-46fd-87c5-66e11cb8f45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:29.329591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ae67ac65-7854-46fd-87c5-66e11cb8f45a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:29.330915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ae67ac65-7854-46fd-87c5-66e11cb8f45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:29.339061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 715/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 02:58:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:29.488409 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50a49d-9a44-476e-90e9-92ccffc97960 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:29.502504 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee50a49d-9a44-476e-90e9-92ccffc97960 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:29.508378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56827aa4-f545-4796-a5ee-bd4c57344f39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:29.509008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56827aa4-f545-4796-a5ee-bd4c57344f39 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:29.509292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56827aa4-f545-4796-a5ee-bd4c57344f39 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:29.509566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56827aa4-f545-4796-a5ee-bd4c57344f39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:29.517643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56827aa4-f545-4796-a5ee-bd4c57344f39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:29.519468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56827aa4-f545-4796-a5ee-bd4c57344f39 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:29.528821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 707/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:30.379064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12ff9-d1e4-4436-8e44-61f889885d8e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:30.379635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12ff9-d1e4-4436-8e44-61f889885d8e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:30.379635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa12ff9-d1e4-4436-8e44-61f889885d8e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:30.381084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 716/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 02:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 02:58:30.385970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c683f-cd5b-44d6-a2c5-522e0a2c0ccf None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:30.386430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c683f-cd5b-44d6-a2c5-522e0a2c0ccf None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:30.386567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c683f-cd5b-44d6-a2c5-522e0a2c0ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:30.404467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-701c683f-cd5b-44d6-a2c5-522e0a2c0ccf None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:30.405598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-701c683f-cd5b-44d6-a2c5-522e0a2c0ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:30.413884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 708/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 02:58:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:30.562036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa12ff9-d1e4-4436-8e44-61f889885d8e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:30.576740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa12ff9-d1e4-4436-8e44-61f889885d8e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:30.582637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee2625-8080-40f6-a1fc-05d27c4d16a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:30.583258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee2625-8080-40f6-a1fc-05d27c4d16a9 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:30.583542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee2625-8080-40f6-a1fc-05d27c4d16a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:30.583819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee2625-8080-40f6-a1fc-05d27c4d16a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:30.591978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee2625-8080-40f6-a1fc-05d27c4d16a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:30.593552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee2625-8080-40f6-a1fc-05d27c4d16a9 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:30.603010 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 717/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:31.455247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe943a-9515-474c-a771-738a5b9f902c None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:31.455815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe943a-9515-474c-a771-738a5b9f902c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:31.455930 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe943a-9515-474c-a771-738a5b9f902c None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:31.457315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 709/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 02:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:31.462109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a714b12-e7bf-4286-b022-5531e46a6d3a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:31.462531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a714b12-e7bf-4286-b022-5531e46a6d3a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:31.462682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a714b12-e7bf-4286-b022-5531e46a6d3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:31.480723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3a714b12-e7bf-4286-b022-5531e46a6d3a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:31.481918 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3a714b12-e7bf-4286-b022-5531e46a6d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:31.490284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 718/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 02:58:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:31.638136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe943a-9515-474c-a771-738a5b9f902c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:31.652481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe943a-9515-474c-a771-738a5b9f902c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:31.657656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5563a8-e35b-4cf6-98e2-1cad57451b1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:31.658331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5563a8-e35b-4cf6-98e2-1cad57451b1d service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:31.658611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5563a8-e35b-4cf6-98e2-1cad57451b1d service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:31.659494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5563a8-e35b-4cf6-98e2-1cad57451b1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:31.667336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5563a8-e35b-4cf6-98e2-1cad57451b1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:31.668885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5563a8-e35b-4cf6-98e2-1cad57451b1d service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:31.678055 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 710/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:32.470214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46870f5-e4ed-4c0c-9d42-4f0adab5e7b3 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:32.470724 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46870f5-e4ed-4c0c-9d42-4f0adab5e7b3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:32.470724 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46870f5-e4ed-4c0c-9d42-4f0adab5e7b3 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:32.472133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 719/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 02:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:32.476877 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534f7a7-505e-403e-9016-c9b84e75ddf2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:32.477316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534f7a7-505e-403e-9016-c9b84e75ddf2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:32.477493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a534f7a7-505e-403e-9016-c9b84e75ddf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:32.495295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a534f7a7-505e-403e-9016-c9b84e75ddf2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:32.496476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a534f7a7-505e-403e-9016-c9b84e75ddf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:32.504957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 711/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 02:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:32.659938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46870f5-e4ed-4c0c-9d42-4f0adab5e7b3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:32.674323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46870f5-e4ed-4c0c-9d42-4f0adab5e7b3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:32.680247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f93de2a-61fa-4a27-915c-f8e02067b388 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:32.680883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93de2a-61fa-4a27-915c-f8e02067b388 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:32.681169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93de2a-61fa-4a27-915c-f8e02067b388 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:32.681450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f93de2a-61fa-4a27-915c-f8e02067b388 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:32.689510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93de2a-61fa-4a27-915c-f8e02067b388 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:32.691759 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f93de2a-61fa-4a27-915c-f8e02067b388 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:32.701049 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 720/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:33.500897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b295d-c7df-4739-8889-b3a88f053a5f None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:33.501469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b295d-c7df-4739-8889-b3a88f053a5f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:33.501469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b295d-c7df-4739-8889-b3a88f053a5f None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:33.503435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 712/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 02:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:33.508282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741fdb-5e02-47e2-a58d-cdb95e7715cb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:33.508691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741fdb-5e02-47e2-a58d-cdb95e7715cb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:33.508843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f741fdb-5e02-47e2-a58d-cdb95e7715cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:33.527040 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8f741fdb-5e02-47e2-a58d-cdb95e7715cb None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:33.528245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8f741fdb-5e02-47e2-a58d-cdb95e7715cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:33.536070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 721/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 02:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:33.686598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b295d-c7df-4739-8889-b3a88f053a5f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:33.701135 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b295d-c7df-4739-8889-b3a88f053a5f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:33.707099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a53d38-291f-4d1c-a59c-127862dfa1da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:33.707732 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53d38-291f-4d1c-a59c-127862dfa1da service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:33.708020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53d38-291f-4d1c-a59c-127862dfa1da service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:33.708297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a53d38-291f-4d1c-a59c-127862dfa1da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:33.716255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a53d38-291f-4d1c-a59c-127862dfa1da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:33.717823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a53d38-291f-4d1c-a59c-127862dfa1da service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:33.727126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 713/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:34.510711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65f631-d3fe-4c94-aa49-decfd0efb205 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:34.511225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65f631-d3fe-4c94-aa49-decfd0efb205 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:34.511225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b65f631-d3fe-4c94-aa49-decfd0efb205 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:34.512578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 722/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 02:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:34.517496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c047e-62c2-4b59-9f4b-6b6cebf47f68 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:34.517934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c047e-62c2-4b59-9f4b-6b6cebf47f68 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:34.518081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c047e-62c2-4b59-9f4b-6b6cebf47f68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:34.536217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-823c047e-62c2-4b59-9f4b-6b6cebf47f68 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:34.537416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-823c047e-62c2-4b59-9f4b-6b6cebf47f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:34.545135 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 714/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 02:58:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:34.694484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65f631-d3fe-4c94-aa49-decfd0efb205 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:34.708839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b65f631-d3fe-4c94-aa49-decfd0efb205 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:34.714028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4401bdd-84bd-4623-952d-c6bf6fbe6d14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:34.714672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4401bdd-84bd-4623-952d-c6bf6fbe6d14 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:34.714952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4401bdd-84bd-4623-952d-c6bf6fbe6d14 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:34.715225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4401bdd-84bd-4623-952d-c6bf6fbe6d14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:34.723751 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4401bdd-84bd-4623-952d-c6bf6fbe6d14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:34.725306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4401bdd-84bd-4623-952d-c6bf6fbe6d14 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:34.734379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 723/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:35.536586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a184fae-bda0-490b-92bb-afc6782c346a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:35.537252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a184fae-bda0-490b-92bb-afc6782c346a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:35.537252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a184fae-bda0-490b-92bb-afc6782c346a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:35.538598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 715/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 02:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:35.543514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689eb1b-4823-4dcb-a0be-f855b612d969 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:35.543934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689eb1b-4823-4dcb-a0be-f855b612d969 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:35.544074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689eb1b-4823-4dcb-a0be-f855b612d969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:35.561977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3689eb1b-4823-4dcb-a0be-f855b612d969 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:35.563229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3689eb1b-4823-4dcb-a0be-f855b612d969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:35.571673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 724/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 02:58:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:35.721752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a184fae-bda0-490b-92bb-afc6782c346a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:35.736106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a184fae-bda0-490b-92bb-afc6782c346a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:35.742069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b70194-ee87-4d85-815c-f3811aa6543e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:35.742719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b70194-ee87-4d85-815c-f3811aa6543e service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:35.743006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b70194-ee87-4d85-815c-f3811aa6543e service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:35.743285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b70194-ee87-4d85-815c-f3811aa6543e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:35.751541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b70194-ee87-4d85-815c-f3811aa6543e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:35.753319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b70194-ee87-4d85-815c-f3811aa6543e service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:35.762605 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 716/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:37.675157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447878a-da03-43f3-bb01-63ec8d8ccf53 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:37.675740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447878a-da03-43f3-bb01-63ec8d8ccf53 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:37.675740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447878a-da03-43f3-bb01-63ec8d8ccf53 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:37.677114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 725/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:37.682276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b5806-4531-49cc-af41-2cf594b21437 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:37.682697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b5806-4531-49cc-af41-2cf594b21437 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:37.682874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b5806-4531-49cc-af41-2cf594b21437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:37.700390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8c7b5806-4531-49cc-af41-2cf594b21437 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:37.701602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8c7b5806-4531-49cc-af41-2cf594b21437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:37.709768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 717/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:37.748312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447878a-da03-43f3-bb01-63ec8d8ccf53 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:37.759361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447878a-da03-43f3-bb01-63ec8d8ccf53 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:37.763020 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c82323c-7d9b-42ab-b23f-51fb3076855d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:37.763604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82323c-7d9b-42ab-b23f-51fb3076855d service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:37.763891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82323c-7d9b-42ab-b23f-51fb3076855d service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:37.764169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c82323c-7d9b-42ab-b23f-51fb3076855d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:37.773919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82323c-7d9b-42ab-b23f-51fb3076855d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:37.775381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c82323c-7d9b-42ab-b23f-51fb3076855d service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:37.785263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 726/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:39.943504 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5994a5-88e5-4724-9daf-4fc1f0d248f2 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:39.944053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5994a5-88e5-4724-9daf-4fc1f0d248f2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:39.944053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5994a5-88e5-4724-9daf-4fc1f0d248f2 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:39.945456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 718/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:39.950782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87164bc1-f1b9-46e9-8d0b-8ad18f786503 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:39.951219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87164bc1-f1b9-46e9-8d0b-8ad18f786503 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:39.951351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87164bc1-f1b9-46e9-8d0b-8ad18f786503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:39.968543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-87164bc1-f1b9-46e9-8d0b-8ad18f786503 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:39.969727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-87164bc1-f1b9-46e9-8d0b-8ad18f786503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:39.977389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 727/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:40.015979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5994a5-88e5-4724-9daf-4fc1f0d248f2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:40.026533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5994a5-88e5-4724-9daf-4fc1f0d248f2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:40.029992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9b7708-34a3-41a9-b1fc-6be8f052a92f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:40.030652 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b7708-34a3-41a9-b1fc-6be8f052a92f service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:40.030941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b7708-34a3-41a9-b1fc-6be8f052a92f service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:40.031222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9b7708-34a3-41a9-b1fc-6be8f052a92f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:40.039975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b7708-34a3-41a9-b1fc-6be8f052a92f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:40.041389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9b7708-34a3-41a9-b1fc-6be8f052a92f service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:40.051082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 719/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:42.370666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b18d5-8345-4d77-ac2a-f326c8689f0d None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:42.371225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b18d5-8345-4d77-ac2a-f326c8689f0d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:42.371225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b18d5-8345-4d77-ac2a-f326c8689f0d None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:42.372602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 728/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:42.377886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcbf4-dd7b-42cf-9dee-57af48077ba0 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:42.378324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcbf4-dd7b-42cf-9dee-57af48077ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:42.378477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872bcbf4-dd7b-42cf-9dee-57af48077ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:42.396065 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-872bcbf4-dd7b-42cf-9dee-57af48077ba0 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:42.397242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-872bcbf4-dd7b-42cf-9dee-57af48077ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:42.404881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 720/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:42.443231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b18d5-8345-4d77-ac2a-f326c8689f0d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:42.453678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b18d5-8345-4d77-ac2a-f326c8689f0d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:42.457763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f298f11-682f-45b7-bc15-1e04f7a1ca2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:42.458414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f298f11-682f-45b7-bc15-1e04f7a1ca2e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:42.458695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f298f11-682f-45b7-bc15-1e04f7a1ca2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:42.458967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f298f11-682f-45b7-bc15-1e04f7a1ca2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:42.467562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f298f11-682f-45b7-bc15-1e04f7a1ca2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:42.468971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f298f11-682f-45b7-bc15-1e04f7a1ca2e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:42.478668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 729/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:44.622303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b281b3a-87a4-43ed-8b2d-63a5f99c87f9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:44.622825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b281b3a-87a4-43ed-8b2d-63a5f99c87f9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:44.622952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b281b3a-87a4-43ed-8b2d-63a5f99c87f9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:44.624520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 721/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:44.631408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30445c17-69aa-4e89-9e2d-f36c3faad8e1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:44.631839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30445c17-69aa-4e89-9e2d-f36c3faad8e1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:44.631972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30445c17-69aa-4e89-9e2d-f36c3faad8e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:44.649541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-30445c17-69aa-4e89-9e2d-f36c3faad8e1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:44.650755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-30445c17-69aa-4e89-9e2d-f36c3faad8e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:44.658468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 730/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 02:58:44.696897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b281b3a-87a4-43ed-8b2d-63a5f99c87f9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:44.707591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b281b3a-87a4-43ed-8b2d-63a5f99c87f9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:44.711259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5285b1-79eb-4fab-b078-f9f6d4b5ace7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:44.711894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5285b1-79eb-4fab-b078-f9f6d4b5ace7 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:44.712174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5285b1-79eb-4fab-b078-f9f6d4b5ace7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:44.712445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5285b1-79eb-4fab-b078-f9f6d4b5ace7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:44.722538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5285b1-79eb-4fab-b078-f9f6d4b5ace7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:44.723984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5285b1-79eb-4fab-b078-f9f6d4b5ace7 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:44.733292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 722/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:46.798774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562162e2-369a-4790-9c04-ad21909e9d04 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:46.799296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562162e2-369a-4790-9c04-ad21909e9d04 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:46.799296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562162e2-369a-4790-9c04-ad21909e9d04 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:46.800677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 731/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:46.805947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d486139-fa23-409a-b41b-4de907796c75 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:46.806402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d486139-fa23-409a-b41b-4de907796c75 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:46.806543 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d486139-fa23-409a-b41b-4de907796c75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:46.824523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7d486139-fa23-409a-b41b-4de907796c75 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:46.825712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7d486139-fa23-409a-b41b-4de907796c75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:46.833962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 723/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:46.871532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562162e2-369a-4790-9c04-ad21909e9d04 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:46.882344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562162e2-369a-4790-9c04-ad21909e9d04 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:46.885941 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575fa2f1-c75c-41f9-aeb9-5eeee1a1aa8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:46.886593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fa2f1-c75c-41f9-aeb9-5eeee1a1aa8b service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:46.886873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fa2f1-c75c-41f9-aeb9-5eeee1a1aa8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:46.887148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575fa2f1-c75c-41f9-aeb9-5eeee1a1aa8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:46.896334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fa2f1-c75c-41f9-aeb9-5eeee1a1aa8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:46.897731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575fa2f1-c75c-41f9-aeb9-5eeee1a1aa8b service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:46.906966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 732/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:48.960655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853183e-3dbe-4cff-be77-24682260cfa4 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:48.961232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853183e-3dbe-4cff-be77-24682260cfa4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:48.961232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853183e-3dbe-4cff-be77-24682260cfa4 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:48.962696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 724/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:48.967722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9907915-fb1e-497b-bc2b-713b64bed897 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:48.968142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9907915-fb1e-497b-bc2b-713b64bed897 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:48.968281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9907915-fb1e-497b-bc2b-713b64bed897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:48.985395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f9907915-fb1e-497b-bc2b-713b64bed897 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:48.987180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f9907915-fb1e-497b-bc2b-713b64bed897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:48.994784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 733/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:49.032493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853183e-3dbe-4cff-be77-24682260cfa4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:49.042905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853183e-3dbe-4cff-be77-24682260cfa4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:49.046375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55872725-9842-4636-b0ea-d6145450303b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:49.047002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55872725-9842-4636-b0ea-d6145450303b service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:49.047284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55872725-9842-4636-b0ea-d6145450303b service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:49.047559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55872725-9842-4636-b0ea-d6145450303b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:49.056660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55872725-9842-4636-b0ea-d6145450303b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:49.058089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55872725-9842-4636-b0ea-d6145450303b service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:49.067935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 725/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:51.225444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149e15-4a7d-42b5-8cd1-6d892283c9c2 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:51.225978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149e15-4a7d-42b5-8cd1-6d892283c9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:51.225978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29149e15-4a7d-42b5-8cd1-6d892283c9c2 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:51.227410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 734/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:51.232501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53d6d-b122-41b4-a195-bbf3af297094 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:51.232882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53d6d-b122-41b4-a195-bbf3af297094 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:51.233075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f53d6d-b122-41b4-a195-bbf3af297094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:51.250760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-05f53d6d-b122-41b4-a195-bbf3af297094 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:51.251957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-05f53d6d-b122-41b4-a195-bbf3af297094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:51.259795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 726/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:51.298320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29149e15-4a7d-42b5-8cd1-6d892283c9c2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:51.308722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29149e15-4a7d-42b5-8cd1-6d892283c9c2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:51.312174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c67f3-1ec0-47cf-a8fb-507103a9263a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:51.312803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c67f3-1ec0-47cf-a8fb-507103a9263a service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:51.313083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c67f3-1ec0-47cf-a8fb-507103a9263a service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:51.313357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c67f3-1ec0-47cf-a8fb-507103a9263a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:51.322136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c67f3-1ec0-47cf-a8fb-507103a9263a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:51.323549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16c67f3-1ec0-47cf-a8fb-507103a9263a service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:51.333186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 735/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:53.384657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2a306-4027-4070-9f2d-77f4afff94a1 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:53.385225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2a306-4027-4070-9f2d-77f4afff94a1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:53.385225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df2a306-4027-4070-9f2d-77f4afff94a1 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:53.386643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 727/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:53.391664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14adf91-6b46-4f32-bcb0-27ac72f05dea None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:53.392063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14adf91-6b46-4f32-bcb0-27ac72f05dea None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:53.392214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14adf91-6b46-4f32-bcb0-27ac72f05dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:53.398221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=79 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:58:53.410742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d14adf91-6b46-4f32-bcb0-27ac72f05dea None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:53.411924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d14adf91-6b46-4f32-bcb0-27ac72f05dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:53.419588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 736/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:53.457150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2a306-4027-4070-9f2d-77f4afff94a1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:53.462458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=73 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:58:53.468827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df2a306-4027-4070-9f2d-77f4afff94a1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:53.472288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4a148d-34a4-492a-aaba-c1c3eb8b7de0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:53.472911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a148d-34a4-492a-aaba-c1c3eb8b7de0 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:53.473191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a148d-34a4-492a-aaba-c1c3eb8b7de0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:53.473463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a148d-34a4-492a-aaba-c1c3eb8b7de0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:53.482062 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a148d-34a4-492a-aaba-c1c3eb8b7de0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:53.483474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4a148d-34a4-492a-aaba-c1c3eb8b7de0 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:53.493015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 728/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:55.521584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997be6ae-5c85-4a0b-9d46-8b76cbd5b0ba None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:55.522218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997be6ae-5c85-4a0b-9d46-8b76cbd5b0ba None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:55.522218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997be6ae-5c85-4a0b-9d46-8b76cbd5b0ba None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:55.523562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 737/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:55.528836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1375684-4660-4b5e-b2aa-8770934cd250 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:55.529263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1375684-4660-4b5e-b2aa-8770934cd250 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:55.529424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1375684-4660-4b5e-b2aa-8770934cd250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:55.547088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a1375684-4660-4b5e-b2aa-8770934cd250 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:55.548271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a1375684-4660-4b5e-b2aa-8770934cd250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:55.556903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 729/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:55.594401 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997be6ae-5c85-4a0b-9d46-8b76cbd5b0ba None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:55.605431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997be6ae-5c85-4a0b-9d46-8b76cbd5b0ba None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:55.608904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef68b08-ffd3-4c02-9fd1-c0582bded9b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:55.609533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef68b08-ffd3-4c02-9fd1-c0582bded9b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:55.609815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef68b08-ffd3-4c02-9fd1-c0582bded9b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:55.610124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef68b08-ffd3-4c02-9fd1-c0582bded9b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:55.618805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef68b08-ffd3-4c02-9fd1-c0582bded9b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:55.620209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef68b08-ffd3-4c02-9fd1-c0582bded9b4 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:55.629631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 738/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:58:57.703912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256eacd-399c-44e1-97d9-a58932037419 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:57.704474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256eacd-399c-44e1-97d9-a58932037419 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:57.704474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c256eacd-399c-44e1-97d9-a58932037419 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:57.705885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 730/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 02:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:58:57.711082 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427af25-204b-4692-bb5e-a6149eba4745 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:57.711490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427af25-204b-4692-bb5e-a6149eba4745 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:57.711636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e427af25-204b-4692-bb5e-a6149eba4745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:57.729976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e427af25-204b-4692-bb5e-a6149eba4745 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:58:57.731319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e427af25-204b-4692-bb5e-a6149eba4745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:58:57.739661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 739/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 02:58:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:58:57.775851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256eacd-399c-44e1-97d9-a58932037419 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:58:57.786935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c256eacd-399c-44e1-97d9-a58932037419 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:58:57.790388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a48b130-6905-4bb7-ae18-70318b5765e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:58:57.791012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48b130-6905-4bb7-ae18-70318b5765e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:58:57.791297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48b130-6905-4bb7-ae18-70318b5765e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:58:57.791569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a48b130-6905-4bb7-ae18-70318b5765e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:58:57.801474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48b130-6905-4bb7-ae18-70318b5765e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:58:57.802913 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a48b130-6905-4bb7-ae18-70318b5765e9 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:58:57.811976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 731/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:58:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:59:07.724349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=11 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:59:07.807898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=14 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:59:17.497403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf330f-8a35-4902-890e-9ad8082c4d1e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:17.497996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf330f-8a35-4902-890e-9ad8082c4d1e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:17.497996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bf330f-8a35-4902-890e-9ad8082c4d1e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:17.499421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 740/1471] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:59:17.506019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26965e-7f1c-4298-9cbc-632e0cf23afe None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:17.506472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26965e-7f1c-4298-9cbc-632e0cf23afe None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:17.506575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26965e-7f1c-4298-9cbc-632e0cf23afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:17.523619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-af26965e-7f1c-4298-9cbc-632e0cf23afe None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:59:17.524824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-af26965e-7f1c-4298-9cbc-632e0cf23afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:59:17.532400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 732/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:59:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 02:59:17.579596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5d61-9104-4bc1-accc-37019ba4ce6e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:17.579902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5d61-9104-4bc1-accc-37019ba4ce6e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:17.580183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a5d61-9104-4bc1-accc-37019ba4ce6e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:17.581613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 741/1473] 10.222.0.13 () {54 vars in 962 bytes} [Sun Jun 26 02:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:59:17.586429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d595135-f236-468e-a98c-61df060e74c6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:17.586720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d595135-f236-468e-a98c-61df060e74c6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:17.586997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d595135-f236-468e-a98c-61df060e74c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:17.604482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1d595135-f236-468e-a98c-61df060e74c6 None None] MFA Rules not processed for user `0dc5b601421a41d1a30a10e86967a920`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:59:17.605787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1d595135-f236-468e-a98c-61df060e74c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:59:17.612077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 733/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Sun Jun 26 02:59:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:59:17.620191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5d61-9104-4bc1-accc-37019ba4ce6e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:59:17.632590 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a5d61-9104-4bc1-accc-37019ba4ce6e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:59:17.636178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf24611f-7518-4dbf-9801-0699ba310839 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0dc5b601421a41d1a30a10e86967a920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:59:17.636809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24611f-7518-4dbf-9801-0699ba310839 service glance] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:17.637090 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24611f-7518-4dbf-9801-0699ba310839 service glance] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:17.637365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf24611f-7518-4dbf-9801-0699ba310839 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:17.646241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24611f-7518-4dbf-9801-0699ba310839 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:59:17.647645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf24611f-7518-4dbf-9801-0699ba310839 service glance] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:59:17.656795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 742/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:59:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:59:19.752719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076517d7-bd7d-4c02-a1e8-d08f2f2b4c02 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:19.753297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076517d7-bd7d-4c02-a1e8-d08f2f2b4c02 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:19.753297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076517d7-bd7d-4c02-a1e8-d08f2f2b4c02 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:19.754741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 734/1476] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:59:19.759816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd1d88-2803-4486-be18-c72fed8aa515 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:19.760264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd1d88-2803-4486-be18-c72fed8aa515 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:19.760380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd1d88-2803-4486-be18-c72fed8aa515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:19.777923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-78bd1d88-2803-4486-be18-c72fed8aa515 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:59:19.779137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-78bd1d88-2803-4486-be18-c72fed8aa515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:59:19.787403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 743/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:59:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:59:19.810108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076517d7-bd7d-4c02-a1e8-d08f2f2b4c02 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:59:19.822299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076517d7-bd7d-4c02-a1e8-d08f2f2b4c02 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:59:19.826587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed5e5a-03a5-485d-97a4-47471f3ce9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:59:19.827217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed5e5a-03a5-485d-97a4-47471f3ce9cf admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:19.827497 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed5e5a-03a5-485d-97a4-47471f3ce9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:19.827766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed5e5a-03a5-485d-97a4-47471f3ce9cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:19.828483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed5e5a-03a5-485d-97a4-47471f3ce9cf admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:59:19.830241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-a9ed5e5a-03a5-485d-97a4-47471f3ce9cf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 02:59:19.830241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 26 02:59:19.830241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: Jun 26 02:59:19.831943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ed5e5a-03a5-485d-97a4-47471f3ce9cf admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:59:19.851373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 735/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Jun 26 02:59:19 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 02:59:21.870745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ec2dd-6b10-4b00-b0e8-1907cf2c372b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:21.871211 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ec2dd-6b10-4b00-b0e8-1907cf2c372b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:21.871308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ec2dd-6b10-4b00-b0e8-1907cf2c372b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:21.872715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 744/1479] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:59:21.878713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e629e-43a5-4322-904d-8929822c2f29 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:21.879140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e629e-43a5-4322-904d-8929822c2f29 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:21.879272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e629e-43a5-4322-904d-8929822c2f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:21.896457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a45e629e-43a5-4322-904d-8929822c2f29 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:59:21.897645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a45e629e-43a5-4322-904d-8929822c2f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:59:21.905244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 736/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:59:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:59:21.948106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ec2dd-6b10-4b00-b0e8-1907cf2c372b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:59:21.962671 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ec2dd-6b10-4b00-b0e8-1907cf2c372b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:59:21.968097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de4d5f-c49d-48d9-b76e-d9e1e96a3004 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:59:21.968726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de4d5f-c49d-48d9-b76e-d9e1e96a3004 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:21.969007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de4d5f-c49d-48d9-b76e-d9e1e96a3004 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:21.969278 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de4d5f-c49d-48d9-b76e-d9e1e96a3004 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:21.977266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de4d5f-c49d-48d9-b76e-d9e1e96a3004 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:59:21.978852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de4d5f-c49d-48d9-b76e-d9e1e96a3004 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:59:21.988416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 745/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:59:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:59:24.156480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c743fc1-79a7-45e3-b556-29d12efd43ec None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:24.157003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c743fc1-79a7-45e3-b556-29d12efd43ec None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:24.157119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c743fc1-79a7-45e3-b556-29d12efd43ec None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:24.158477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 737/1482] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:59:24.164264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4fb51-00a8-4095-a0c2-45ee071125f7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:24.164678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4fb51-00a8-4095-a0c2-45ee071125f7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:24.164813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a4fb51-00a8-4095-a0c2-45ee071125f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:24.182270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-08a4fb51-00a8-4095-a0c2-45ee071125f7 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:59:24.183585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-08a4fb51-00a8-4095-a0c2-45ee071125f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:59:24.191318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 746/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:59:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:59:24.227567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c743fc1-79a7-45e3-b556-29d12efd43ec None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:59:24.238131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c743fc1-79a7-45e3-b556-29d12efd43ec None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:59:24.241567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e788b27d-10ab-4af0-a0b5-7fb2c4eb63f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:59:24.242231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788b27d-10ab-4af0-a0b5-7fb2c4eb63f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:24.242518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788b27d-10ab-4af0-a0b5-7fb2c4eb63f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:24.242795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e788b27d-10ab-4af0-a0b5-7fb2c4eb63f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:24.251714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788b27d-10ab-4af0-a0b5-7fb2c4eb63f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:59:24.253126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e788b27d-10ab-4af0-a0b5-7fb2c4eb63f6 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:59:24.262978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 738/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:59:26.324505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29866038-98fd-43cb-b85c-cb7be84a5063 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:26.325076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29866038-98fd-43cb-b85c-cb7be84a5063 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:26.325076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29866038-98fd-43cb-b85c-cb7be84a5063 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:26.326523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 747/1485] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:59:26.332248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602d86a-a208-43c4-b31f-40d5c2a4c32b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:26.332654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602d86a-a208-43c4-b31f-40d5c2a4c32b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:26.332808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f602d86a-a208-43c4-b31f-40d5c2a4c32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:26.349804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f602d86a-a208-43c4-b31f-40d5c2a4c32b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:59:26.351013 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f602d86a-a208-43c4-b31f-40d5c2a4c32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:59:26.358682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 739/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:59:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:59:26.396295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29866038-98fd-43cb-b85c-cb7be84a5063 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:59:26.406780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29866038-98fd-43cb-b85c-cb7be84a5063 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:59:26.410341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b905951-ab1d-4298-b743-77c6ce326ea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:59:26.410966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b905951-ab1d-4298-b743-77c6ce326ea6 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:26.411249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b905951-ab1d-4298-b743-77c6ce326ea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:26.411521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b905951-ab1d-4298-b743-77c6ce326ea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:26.420169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b905951-ab1d-4298-b743-77c6ce326ea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:59:26.421624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b905951-ab1d-4298-b743-77c6ce326ea6 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:59:26.431448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 748/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:59:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:59:28.617032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdbf55-c588-4548-8d22-9289802a510a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:28.617593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdbf55-c588-4548-8d22-9289802a510a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:28.617593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdbf55-c588-4548-8d22-9289802a510a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:28.618997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 740/1488] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:59:28.624289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e565a-c77e-43fc-91ba-0f348e6b06d6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:28.624703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e565a-c77e-43fc-91ba-0f348e6b06d6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:28.625097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91e565a-c77e-43fc-91ba-0f348e6b06d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:28.642997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d91e565a-c77e-43fc-91ba-0f348e6b06d6 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:59:28.644688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d91e565a-c77e-43fc-91ba-0f348e6b06d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:59:28.652498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 749/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:59:28.692868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdbf55-c588-4548-8d22-9289802a510a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:59:28.703562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdbf55-c588-4548-8d22-9289802a510a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:59:28.707486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4a0e76-a9de-4513-82f0-ba30ee1219fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:59:28.708131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a0e76-a9de-4513-82f0-ba30ee1219fd service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:28.708413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a0e76-a9de-4513-82f0-ba30ee1219fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:28.708684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4a0e76-a9de-4513-82f0-ba30ee1219fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:28.717339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a0e76-a9de-4513-82f0-ba30ee1219fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:59:28.718777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4a0e76-a9de-4513-82f0-ba30ee1219fd service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:59:28.728582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 741/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:59:31.029378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210697f-85d7-44be-99ff-733ddfaefe61 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:31.029949 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210697f-85d7-44be-99ff-733ddfaefe61 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:31.029949 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e210697f-85d7-44be-99ff-733ddfaefe61 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:31.031640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 750/1491] 10.222.0.13 () {54 vars in 926 bytes} [Sun Jun 26 02:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 02:59:31.037551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad68c4d-285b-48cc-a69a-7d0fb592db6f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:31.037962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad68c4d-285b-48cc-a69a-7d0fb592db6f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:31.038159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad68c4d-285b-48cc-a69a-7d0fb592db6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:31.055255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0ad68c4d-285b-48cc-a69a-7d0fb592db6f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 02:59:31.056432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0ad68c4d-285b-48cc-a69a-7d0fb592db6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 02:59:31.064006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 742/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Sun Jun 26 02:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 02:59:31.103685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210697f-85d7-44be-99ff-733ddfaefe61 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 02:59:31.114124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e210697f-85d7-44be-99ff-733ddfaefe61 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 02:59:31.117562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a624440-327b-4208-92c5-ac38e78e862d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 02:59:31.118229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a624440-327b-4208-92c5-ac38e78e862d service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 02:59:31.118515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a624440-327b-4208-92c5-ac38e78e862d service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 02:59:31.118795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a624440-327b-4208-92c5-ac38e78e862d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 02:59:31.127823 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a624440-327b-4208-92c5-ac38e78e862d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 02:59:31.129247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a624440-327b-4208-92c5-ac38e78e862d service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 02:59:31.138522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 751/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 02:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 02:59:41.050235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=33 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 02:59:41.134333 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=33 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:00:56.676219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46381dd0-62b1-4530-a643-2a173d56088e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:56.676802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46381dd0-62b1-4530-a643-2a173d56088e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:56.676802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46381dd0-62b1-4530-a643-2a173d56088e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:56.694651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-46381dd0-62b1-4530-a643-2a173d56088e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:00:56.695837 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-46381dd0-62b1-4530-a643-2a173d56088e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:00:56.703862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 743/1494] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:00:56.712318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e7bea-22e4-4184-ba29-ee9f48d8dd6a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:56.712785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e7bea-22e4-4184-ba29-ee9f48d8dd6a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:56.712904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e7bea-22e4-4184-ba29-ee9f48d8dd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:56.729993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-770e7bea-22e4-4184-ba29-ee9f48d8dd6a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:00:56.731191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-770e7bea-22e4-4184-ba29-ee9f48d8dd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:00:56.738769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 752/1495] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:00:56.742940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46381dd0-62b1-4530-a643-2a173d56088e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:00:56.753093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46381dd0-62b1-4530-a643-2a173d56088e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:00:56.757507 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8dd38-41a0-4ecc-b7ad-ab0c95c5eba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:00:56.758177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8dd38-41a0-4ecc-b7ad-ab0c95c5eba3 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:56.758465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8dd38-41a0-4ecc-b7ad-ab0c95c5eba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:56.758742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d8dd38-41a0-4ecc-b7ad-ab0c95c5eba3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:56.759460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8dd38-41a0-4ecc-b7ad-ab0c95c5eba3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:00:56.759769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8dd38-41a0-4ecc-b7ad-ab0c95c5eba3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:00:56.760770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-e0d8dd38-41a0-4ecc-b7ad-ab0c95c5eba3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:00:56.760770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 26 03:00:56.760770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: Jun 26 03:00:56.761641 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d8dd38-41a0-4ecc-b7ad-ab0c95c5eba3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:00:56.773515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 744/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:00:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:00:56.781179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e7bea-22e4-4184-ba29-ee9f48d8dd6a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:00:56.790346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e7bea-22e4-4184-ba29-ee9f48d8dd6a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:00:56.794782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983e122-1dcb-4181-bf9a-25288a446fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:00:56.795442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983e122-1dcb-4181-bf9a-25288a446fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:56.795735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983e122-1dcb-4181-bf9a-25288a446fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:56.795988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5983e122-1dcb-4181-bf9a-25288a446fe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:56.796671 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983e122-1dcb-4181-bf9a-25288a446fe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:00:56.798476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-5983e122-1dcb-4181-bf9a-25288a446fe2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:00:56.798476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 26 03:00:56.798476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: Jun 26 03:00:56.799667 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5983e122-1dcb-4181-bf9a-25288a446fe2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:00:56.834995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 753/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:00:56 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:00:56.839133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8dd38-41a0-4ecc-b7ad-ab0c95c5eba3 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:00:56.849113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d8dd38-41a0-4ecc-b7ad-ab0c95c5eba3 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:00:56.905057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba416c-5707-4125-85e7-e3853fda9d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:00:56.905721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba416c-5707-4125-85e7-e3853fda9d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:56.906012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba416c-5707-4125-85e7-e3853fda9d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:56.906324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba416c-5707-4125-85e7-e3853fda9d60 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:56.907024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba416c-5707-4125-85e7-e3853fda9d60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:00:56.908106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-c5ba416c-5707-4125-85e7-e3853fda9d60 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:00:56.908106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 26 03:00:56.908106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: Jun 26 03:00:56.909350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba416c-5707-4125-85e7-e3853fda9d60 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:00:56.929131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 745/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:00:56 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:00:56.933188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983e122-1dcb-4181-bf9a-25288a446fe2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:00:56.943621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5983e122-1dcb-4181-bf9a-25288a446fe2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:00:56.948163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94907f1a-5b59-434b-bf4b-408053f7a0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:00:56.948815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94907f1a-5b59-434b-bf4b-408053f7a0ef admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:56.949100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94907f1a-5b59-434b-bf4b-408053f7a0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:56.949380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94907f1a-5b59-434b-bf4b-408053f7a0ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:56.950127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94907f1a-5b59-434b-bf4b-408053f7a0ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:00:56.951152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-94907f1a-5b59-434b-bf4b-408053f7a0ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:00:56.951152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 26 03:00:56.951152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: Jun 26 03:00:56.952029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94907f1a-5b59-434b-bf4b-408053f7a0ef admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:00:56.965457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 754/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:00:56.969219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba416c-5707-4125-85e7-e3853fda9d60 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:00:56.978989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba416c-5707-4125-85e7-e3853fda9d60 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:00:56.983496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe17c3-56f8-4fe9-802d-6feb42d8247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:00:56.984137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe17c3-56f8-4fe9-802d-6feb42d8247f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:56.984420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe17c3-56f8-4fe9-802d-6feb42d8247f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:56.984691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe17c3-56f8-4fe9-802d-6feb42d8247f admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:56.985408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe17c3-56f8-4fe9-802d-6feb42d8247f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:00:56.986532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-d8fe17c3-56f8-4fe9-802d-6feb42d8247f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:00:56.986532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 26 03:00:56.986532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: Jun 26 03:00:56.987424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe17c3-56f8-4fe9-802d-6feb42d8247f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:00:57.002464 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 746/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:00:57.007094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94907f1a-5b59-434b-bf4b-408053f7a0ef admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:00:57.016765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94907f1a-5b59-434b-bf4b-408053f7a0ef admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:00:57.021747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a679cd-3b3e-4089-9814-ca23e0a5a1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:00:57.022661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a679cd-3b3e-4089-9814-ca23e0a5a1af admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:57.022949 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a679cd-3b3e-4089-9814-ca23e0a5a1af admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:57.023227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a679cd-3b3e-4089-9814-ca23e0a5a1af admin admin] PATH_INFO: `/v3/projects/bc0e84e8dea04167b4a29151a8665ef2/users/b4435067a793404b860c2b36a56a324c/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:57.037487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a679cd-3b3e-4089-9814-ca23e0a5a1af admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc0e84e8dea04167b4a29151a8665ef2, user_id=b4435067a793404b860c2b36a56a324c, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:00:57.038547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-f3a679cd-3b3e-4089-9814-ca23e0a5a1af admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:00:57.038547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 26 03:00:57.038547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: Jun 26 03:00:57.040448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a679cd-3b3e-4089-9814-ca23e0a5a1af admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:00:57.052714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 755/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:00:57 2022] PUT /identity/v3/projects/bc0e84e8dea04167b4a29151a8665ef2/users/b4435067a793404b860c2b36a56a324c/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:00:57.064636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697de179-7c4a-480f-8bbd-8011c9397664 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:57.064933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697de179-7c4a-480f-8bbd-8011c9397664 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:57.065212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697de179-7c4a-480f-8bbd-8011c9397664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:57.092100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-697de179-7c4a-480f-8bbd-8011c9397664 None None] MFA Rules not processed for user `b4435067a793404b860c2b36a56a324c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:00:57.093285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-697de179-7c4a-480f-8bbd-8011c9397664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:00:57.153940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 747/1502] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:00:57 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:00:57.167054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4dbfa-d62b-4e3c-ae47-b254fe2a9791 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:57.167346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4dbfa-d62b-4e3c-ae47-b254fe2a9791 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:57.167625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4dbfa-d62b-4e3c-ae47-b254fe2a9791 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:57.168995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 756/1503] 10.222.0.13 () {54 vars in 963 bytes} [Sun Jun 26 03:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:00:57.174197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21f047-c344-405d-b9c1-a2c8cfd4d424 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:57.174488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21f047-c344-405d-b9c1-a2c8cfd4d424 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:57.174766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a21f047-c344-405d-b9c1-a2c8cfd4d424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:57.192460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9a21f047-c344-405d-b9c1-a2c8cfd4d424 None None] MFA Rules not processed for user `5e65f36f772b4220a909f8fda83ba42e`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:00:57.193649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9a21f047-c344-405d-b9c1-a2c8cfd4d424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:00:57.234719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:00:57.243191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 748/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Sun Jun 26 03:00:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:00:57.250749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4dbfa-d62b-4e3c-ae47-b254fe2a9791 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:00:57.261710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4dbfa-d62b-4e3c-ae47-b254fe2a9791 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:00:57.265184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddb542-2d60-41b5-a565-bce94629cff5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:00:57.265806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddb542-2d60-41b5-a565-bce94629cff5 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:57.266112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddb542-2d60-41b5-a565-bce94629cff5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:57.266391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ddb542-2d60-41b5-a565-bce94629cff5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:57.274420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddb542-2d60-41b5-a565-bce94629cff5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:00:57.275827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ddb542-2d60-41b5-a565-bce94629cff5 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:00:57.284744 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 757/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:00:57 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:00:57.326530 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982eb85-264e-4b90-8f7b-4648a936afd2 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:57.326821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982eb85-264e-4b90-8f7b-4648a936afd2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:57.327099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982eb85-264e-4b90-8f7b-4648a936afd2 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:57.328414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 749/1506] 10.222.0.13 () {52 vars in 829 bytes} [Sun Jun 26 03:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:00:57.331885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddb542-2d60-41b5-a565-bce94629cff5 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:00:57.392616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ddb542-2d60-41b5-a565-bce94629cff5 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:00:57.397355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:00:57.398232 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:57.398524 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:57.398800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] PATH_INFO: `/v3/users/b4435067a793404b860c2b36a56a324c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:57.399969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4435067a793404b860c2b36a56a324c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:00:57.400971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:00:57.400971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 26 03:00:57.400971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: Jun 26 03:00:57.402213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:00:57.464735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4435067a793404b860c2b36a56a324c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:00:57.465473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4435067a793404b860c2b36a56a324c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:00:57.478405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4435067a793404b860c2b36a56a324c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:00:57.487708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4435067a793404b860c2b36a56a324c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:00:57.488417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4435067a793404b860c2b36a56a324c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:00:57.505200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:00:57.514967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 758/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:00:57 2022] DELETE /identity/v3/users/b4435067a793404b860c2b36a56a324c => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:00:57.519321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7a1ab-4457-4a34-ab94-775efcdc8a95 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:57.519617 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7a1ab-4457-4a34-ab94-775efcdc8a95 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:57.519896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb7a1ab-4457-4a34-ab94-775efcdc8a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:57.537129 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-deb7a1ab-4457-4a34-ab94-775efcdc8a95 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:00:57.538359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-deb7a1ab-4457-4a34-ab94-775efcdc8a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:00:57.594829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 750/1508] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:00:57.605462 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:00:57.606938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:00:57.672558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1394d81-ac61-4062-a828-5ecc8f94de42 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:00:57.676708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846f69c4-181f-4715-9760-051e7d4481e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:00:57.677352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f69c4-181f-4715-9760-051e7d4481e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:57.677631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f69c4-181f-4715-9760-051e7d4481e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:57.677902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846f69c4-181f-4715-9760-051e7d4481e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:57.686531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f69c4-181f-4715-9760-051e7d4481e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:00:57.687979 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846f69c4-181f-4715-9760-051e7d4481e4 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:00:57.697153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 759/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:00:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:00:58.041187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7a1ab-4457-4a34-ab94-775efcdc8a95 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:00:58.042802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-deb7a1ab-4457-4a34-ab94-775efcdc8a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:00:58.054899 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb7a1ab-4457-4a34-ab94-775efcdc8a95 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:00:58.059563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:00:58.060425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:00:58.060738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:00:58.061014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] PATH_INFO: `/v3/projects/bc0e84e8dea04167b4a29151a8665ef2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:00:58.062221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc0e84e8dea04167b4a29151a8665ef2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:00:58.063238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:00:58.063238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 26 03:00:58.063238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: Jun 26 03:00:58.064437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:00:58.127327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc0e84e8dea04167b4a29151a8665ef2', 'request_id': 'req-463fe35d-ae0b-4ce6-b930-32809c34f411'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:00:58.136922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc0e84e8dea04167b4a29151a8665ef2', 'request_id': 'req-463fe35d-ae0b-4ce6-b930-32809c34f411'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:00:58.143796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc0e84e8dea04167b4a29151a8665ef2', 'request_id': 'req-463fe35d-ae0b-4ce6-b930-32809c34f411'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:00:58.162995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] The token cache is being invalidate because project bc0e84e8dea04167b4a29151a8665ef2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:00:58.163284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:00:58.163944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-463fe35d-ae0b-4ce6-b930-32809c34f411 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:00:58.165567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 751/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:00:58 2022] DELETE /identity/v3/projects/bc0e84e8dea04167b4a29151a8665ef2 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:01:07.693065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:01:08.160029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:02:33.382515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefb3c6-d7e5-4b99-92bd-8366f58fb2b9 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:33.383107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefb3c6-d7e5-4b99-92bd-8366f58fb2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:33.383107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfefb3c6-d7e5-4b99-92bd-8366f58fb2b9 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:33.384498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 760/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 03:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:02:33.389379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de0c1e-d9c2-425b-afc4-96e4a9a1d07c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:33.389846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de0c1e-d9c2-425b-afc4-96e4a9a1d07c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:33.389943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de0c1e-d9c2-425b-afc4-96e4a9a1d07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:33.408751 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-54de0c1e-d9c2-425b-afc4-96e4a9a1d07c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:02:33.409956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-54de0c1e-d9c2-425b-afc4-96e4a9a1d07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:33.468844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 752/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 03:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:02:33.621665 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefb3c6-d7e5-4b99-92bd-8366f58fb2b9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:02:33.623395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cfefb3c6-d7e5-4b99-92bd-8366f58fb2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:33.698123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfefb3c6-d7e5-4b99-92bd-8366f58fb2b9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:02:33.704045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd093f13-90cb-4511-ba5b-2570f4c28706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:02:33.704685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd093f13-90cb-4511-ba5b-2570f4c28706 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:33.704968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd093f13-90cb-4511-ba5b-2570f4c28706 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:33.705243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd093f13-90cb-4511-ba5b-2570f4c28706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:33.713125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd093f13-90cb-4511-ba5b-2570f4c28706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:02:33.714717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd093f13-90cb-4511-ba5b-2570f4c28706 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:02:33.723968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 761/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:02:34.541791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ed608-6aff-41cf-97f8-a40c670c3580 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:34.542371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ed608-6aff-41cf-97f8-a40c670c3580 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:34.542371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3ed608-6aff-41cf-97f8-a40c670c3580 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:34.543721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 753/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 03:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:02:34.548932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba18009-1e2a-475d-b56d-a06797fe163a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:34.549362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba18009-1e2a-475d-b56d-a06797fe163a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:34.549516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba18009-1e2a-475d-b56d-a06797fe163a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:34.567433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-fba18009-1e2a-475d-b56d-a06797fe163a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:02:34.568609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fba18009-1e2a-475d-b56d-a06797fe163a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:34.576238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 762/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 03:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:02:34.741667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ed608-6aff-41cf-97f8-a40c670c3580 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:02:34.756004 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3ed608-6aff-41cf-97f8-a40c670c3580 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:02:34.761570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70343247-1e98-4c12-a90e-de0722839287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:02:34.762237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70343247-1e98-4c12-a90e-de0722839287 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:34.762531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70343247-1e98-4c12-a90e-de0722839287 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:34.762808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70343247-1e98-4c12-a90e-de0722839287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:34.771472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70343247-1e98-4c12-a90e-de0722839287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:02:34.773059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70343247-1e98-4c12-a90e-de0722839287 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:02:34.783001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 754/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:02:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:02:35.590182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd83b-7251-44a5-9a2d-0cc86a1ec6c8 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:35.590752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd83b-7251-44a5-9a2d-0cc86a1ec6c8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:35.590752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93cd83b-7251-44a5-9a2d-0cc86a1ec6c8 None None] PATH_INFO: `/v3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:35.592053 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 763/1517] 10.222.0.43 () {54 vars in 925 bytes} [Sun Jun 26 03:02:35 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:02:35.598949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e146a16-4a96-4b8d-b7a2-cd269d515bb8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:35.599352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e146a16-4a96-4b8d-b7a2-cd269d515bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:35.599518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e146a16-4a96-4b8d-b7a2-cd269d515bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:35.617701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6e146a16-4a96-4b8d-b7a2-cd269d515bb8 None None] MFA Rules not processed for user `19ce6f259c22441486bfc04811a271b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:02:35.618935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6e146a16-4a96-4b8d-b7a2-cd269d515bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:35.680886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 755/1518] 10.222.0.43 () {58 vars in 1015 bytes} [Sun Jun 26 03:02:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:02:44.562395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=37 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:02:44.896648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a846b-54a9-48ca-ab7d-f07207756769 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:44.896940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a846b-54a9-48ca-ab7d-f07207756769 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:44.897222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651a846b-54a9-48ca-ab7d-f07207756769 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:44.898696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 764/1519] 10.222.0.13 () {54 vars in 960 bytes} [Sun Jun 26 03:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:02:44.904925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f813100-c99c-4f2c-bd99-240fbd656944 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:44.905414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f813100-c99c-4f2c-bd99-240fbd656944 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:44.905519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f813100-c99c-4f2c-bd99-240fbd656944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:44.924146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7f813100-c99c-4f2c-bd99-240fbd656944 None None] MFA Rules not processed for user `19ce6f259c22441486bfc04811a271b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:02:44.925372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7f813100-c99c-4f2c-bd99-240fbd656944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:44.933840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 756/1520] 10.222.0.13 () {58 vars in 1057 bytes} [Sun Jun 26 03:02:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:02:44.942032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a846b-54a9-48ca-ab7d-f07207756769 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:02:44.955151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651a846b-54a9-48ca-ab7d-f07207756769 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:02:44.960203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07fccfe-ebea-4e7f-9535-13ba7b1d7349 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:02:44.960839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fccfe-ebea-4e7f-9535-13ba7b1d7349 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:44.961131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fccfe-ebea-4e7f-9535-13ba7b1d7349 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:44.961414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07fccfe-ebea-4e7f-9535-13ba7b1d7349 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:44.969847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fccfe-ebea-4e7f-9535-13ba7b1d7349 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:02:44.971455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07fccfe-ebea-4e7f-9535-13ba7b1d7349 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:02:44.980897 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 765/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:02:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:02:46.006902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75908532-0bd2-4185-b478-f7b320672ed9 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:46.007442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75908532-0bd2-4185-b478-f7b320672ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:46.007442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75908532-0bd2-4185-b478-f7b320672ed9 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:46.008883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 757/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 03:02:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:02:46.013590 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd31dc5-325e-4583-a25a-7a415d3b0d7b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:46.014009 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd31dc5-325e-4583-a25a-7a415d3b0d7b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:46.014182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd31dc5-325e-4583-a25a-7a415d3b0d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:46.032215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ddd31dc5-325e-4583-a25a-7a415d3b0d7b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:02:46.033396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ddd31dc5-325e-4583-a25a-7a415d3b0d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:46.040955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 766/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 03:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:02:46.193082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75908532-0bd2-4185-b478-f7b320672ed9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:02:46.207417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75908532-0bd2-4185-b478-f7b320672ed9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:02:46.213142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8543eee-e551-4b32-a67b-9bf09c011f77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:02:46.213793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8543eee-e551-4b32-a67b-9bf09c011f77 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:46.214110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8543eee-e551-4b32-a67b-9bf09c011f77 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:46.214400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8543eee-e551-4b32-a67b-9bf09c011f77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:46.222926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8543eee-e551-4b32-a67b-9bf09c011f77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:02:46.224487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8543eee-e551-4b32-a67b-9bf09c011f77 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:02:46.234223 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 758/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:02:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:02:47.062245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427b061-465d-4e95-8146-af84250823c6 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:47.062725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427b061-465d-4e95-8146-af84250823c6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:47.062855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427b061-465d-4e95-8146-af84250823c6 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:47.064224 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 767/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jun 26 03:02:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:02:47.069028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4410-331b-4418-ae51-689ee3cf18f2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:47.069409 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4410-331b-4418-ae51-689ee3cf18f2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:47.069594 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d4410-331b-4418-ae51-689ee3cf18f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:47.087810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-884d4410-331b-4418-ae51-689ee3cf18f2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:02:47.089015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-884d4410-331b-4418-ae51-689ee3cf18f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:47.097381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 759/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jun 26 03:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:02:47.247426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427b061-465d-4e95-8146-af84250823c6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:02:47.261643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427b061-465d-4e95-8146-af84250823c6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:02:47.267756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ea9e5d-44f8-4eba-959d-65a0756d944a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:02:47.268390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea9e5d-44f8-4eba-959d-65a0756d944a service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:47.268674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea9e5d-44f8-4eba-959d-65a0756d944a service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:47.268975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ea9e5d-44f8-4eba-959d-65a0756d944a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:47.277134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea9e5d-44f8-4eba-959d-65a0756d944a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:02:47.278733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ea9e5d-44f8-4eba-959d-65a0756d944a service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:02:47.288563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 768/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:02:51.146848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008e66e-d51f-417c-974a-f5a53a696f5e None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:51.147453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008e66e-d51f-417c-974a-f5a53a696f5e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:51.147453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008e66e-d51f-417c-974a-f5a53a696f5e None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:51.148871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 760/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 03:02:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:02:51.155292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54377f8-5316-4192-ab64-68b4699f89db None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:51.155793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54377f8-5316-4192-ab64-68b4699f89db None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:51.155893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54377f8-5316-4192-ab64-68b4699f89db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:51.174685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e54377f8-5316-4192-ab64-68b4699f89db None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:02:51.175876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e54377f8-5316-4192-ab64-68b4699f89db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:51.183861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 769/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 03:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:02:51.221000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008e66e-d51f-417c-974a-f5a53a696f5e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:02:51.222692 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2008e66e-d51f-417c-974a-f5a53a696f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:51.278433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008e66e-d51f-417c-974a-f5a53a696f5e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:02:51.282462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d84996-b4e3-439f-a413-dcc4cd8534b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:02:51.283103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84996-b4e3-439f-a413-dcc4cd8534b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:51.283396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84996-b4e3-439f-a413-dcc4cd8534b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:51.283694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d84996-b4e3-439f-a413-dcc4cd8534b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:51.292664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d84996-b4e3-439f-a413-dcc4cd8534b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:02:51.294110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d84996-b4e3-439f-a413-dcc4cd8534b0 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:02:51.299574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:02:51.304587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 761/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:02:53.311176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a880de4-6cab-461c-8187-4180845c9dac None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:53.311738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a880de4-6cab-461c-8187-4180845c9dac None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:53.311738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a880de4-6cab-461c-8187-4180845c9dac None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:53.313678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 770/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 03:02:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:02:53.318999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff2cb-10b1-4ca3-87b7-06eb7f723d74 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:53.319419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff2cb-10b1-4ca3-87b7-06eb7f723d74 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:53.319561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404ff2cb-10b1-4ca3-87b7-06eb7f723d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:53.337801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-404ff2cb-10b1-4ca3-87b7-06eb7f723d74 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:02:53.339033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-404ff2cb-10b1-4ca3-87b7-06eb7f723d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:53.346710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 762/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 03:02:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:02:53.381701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a880de4-6cab-461c-8187-4180845c9dac None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:02:53.392720 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a880de4-6cab-461c-8187-4180845c9dac None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:02:53.396193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b86d4ff-aad9-4331-b7f6-c6c23d7b5c5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:02:53.396817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86d4ff-aad9-4331-b7f6-c6c23d7b5c5b service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:53.397100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86d4ff-aad9-4331-b7f6-c6c23d7b5c5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:53.397377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b86d4ff-aad9-4331-b7f6-c6c23d7b5c5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:53.406146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86d4ff-aad9-4331-b7f6-c6c23d7b5c5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:02:53.407555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b86d4ff-aad9-4331-b7f6-c6c23d7b5c5b service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:02:53.417041 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 771/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:02:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:02:57.091240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fc835-3c6e-4594-99ac-cd8e54ee1ce3 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:57.091791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fc835-3c6e-4594-99ac-cd8e54ee1ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:57.091900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839fc835-3c6e-4594-99ac-cd8e54ee1ce3 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:57.093241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 763/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 03:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:02:57.099180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f559dc4-e5f0-4589-8975-f231649f2324 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:57.099608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f559dc4-e5f0-4589-8975-f231649f2324 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:57.099746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f559dc4-e5f0-4589-8975-f231649f2324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:57.117999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1f559dc4-e5f0-4589-8975-f231649f2324 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:02:57.119202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1f559dc4-e5f0-4589-8975-f231649f2324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:57.126889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 772/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 03:02:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:02:57.425950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fc835-3c6e-4594-99ac-cd8e54ee1ce3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:02:57.427663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-839fc835-3c6e-4594-99ac-cd8e54ee1ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:57.483084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839fc835-3c6e-4594-99ac-cd8e54ee1ce3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:02:57.486725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949415d4-5364-43ef-9df7-d0c9db641799 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:02:57.487360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949415d4-5364-43ef-9df7-d0c9db641799 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:57.487645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949415d4-5364-43ef-9df7-d0c9db641799 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:57.487921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949415d4-5364-43ef-9df7-d0c9db641799 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:57.496537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949415d4-5364-43ef-9df7-d0c9db641799 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:02:57.497952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949415d4-5364-43ef-9df7-d0c9db641799 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:02:57.507394 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 764/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:02:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:02:59.545471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16779a13-b4f3-4d8b-9d7e-f9a6c9d481c5 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:59.546016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16779a13-b4f3-4d8b-9d7e-f9a6c9d481c5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:59.546016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16779a13-b4f3-4d8b-9d7e-f9a6c9d481c5 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:59.547668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 773/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jun 26 03:02:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:02:59.552587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39299158-1c74-4a6f-a2e2-679456b6af24 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:59.553008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39299158-1c74-4a6f-a2e2-679456b6af24 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:59.553156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39299158-1c74-4a6f-a2e2-679456b6af24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:59.571293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-39299158-1c74-4a6f-a2e2-679456b6af24 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:02:59.572483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-39299158-1c74-4a6f-a2e2-679456b6af24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:59.580842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 765/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jun 26 03:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:02:59.879622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16779a13-b4f3-4d8b-9d7e-f9a6c9d481c5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:02:59.881295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-16779a13-b4f3-4d8b-9d7e-f9a6c9d481c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:02:59.894141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16779a13-b4f3-4d8b-9d7e-f9a6c9d481c5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:02:59.897624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee8ce0-69f1-4858-97b3-09c6d1f0b298 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:02:59.898289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee8ce0-69f1-4858-97b3-09c6d1f0b298 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:02:59.898573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee8ce0-69f1-4858-97b3-09c6d1f0b298 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:02:59.898848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee8ce0-69f1-4858-97b3-09c6d1f0b298 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:02:59.907432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee8ce0-69f1-4858-97b3-09c6d1f0b298 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:02:59.908845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ee8ce0-69f1-4858-97b3-09c6d1f0b298 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:02:59.917990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 774/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:03:09.566159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=32 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:03:09.913958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=33 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:28.158249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5298e-1e55-4e69-8366-d9b13389914e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.159376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5298e-1e55-4e69-8366-d9b13389914e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.159905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a5298e-1e55-4e69-8366-d9b13389914e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.189639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-41a5298e-1e55-4e69-8366-d9b13389914e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.191256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-41a5298e-1e55-4e69-8366-d9b13389914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.200111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 766/1540] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:28.209127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917472c-7763-43b5-967d-12c0947d2cb8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.209745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917472c-7763-43b5-967d-12c0947d2cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.209745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917472c-7763-43b5-967d-12c0947d2cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.236765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a917472c-7763-43b5-967d-12c0947d2cb8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.237990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a917472c-7763-43b5-967d-12c0947d2cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.252438 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 775/1541] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:04:28.257876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5298e-1e55-4e69-8366-d9b13389914e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:28.273087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a5298e-1e55-4e69-8366-d9b13389914e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:28.282740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb5fa3-f07e-4e84-a6ce-264b805d8932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:28.283704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5fa3-f07e-4e84-a6ce-264b805d8932 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.284214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5fa3-f07e-4e84-a6ce-264b805d8932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.284721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb5fa3-f07e-4e84-a6ce-264b805d8932 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.285688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb5fa3-f07e-4e84-a6ce-264b805d8932 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:04:28.286242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb5fa3-f07e-4e84-a6ce-264b805d8932 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:28.288056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb5fa3-f07e-4e84-a6ce-264b805d8932 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:28.306769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 767/1542] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:04:28.315379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917472c-7763-43b5-967d-12c0947d2cb8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:28.325960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917472c-7763-43b5-967d-12c0947d2cb8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:28.335881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1677ee84-79f7-41a7-a136-e29329a04cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:28.346951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677ee84-79f7-41a7-a136-e29329a04cbc admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.346951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677ee84-79f7-41a7-a136-e29329a04cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.346951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1677ee84-79f7-41a7-a136-e29329a04cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.346951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677ee84-79f7-41a7-a136-e29329a04cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:28.346951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1677ee84-79f7-41a7-a136-e29329a04cbc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:28.392377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 776/1543] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/projects => generated 401 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:28.397024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb5fa3-f07e-4e84-a6ce-264b805d8932 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:28.415545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb5fa3-f07e-4e84-a6ce-264b805d8932 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:28.471086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08c0fa-2c07-4bac-8de4-df4ab7e8eca9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.471396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08c0fa-2c07-4bac-8de4-df4ab7e8eca9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.471666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08c0fa-2c07-4bac-8de4-df4ab7e8eca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.493850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c347193-4179-4c9d-a61a-585bf27153ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:28.494563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c347193-4179-4c9d-a61a-585bf27153ba admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.494849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c347193-4179-4c9d-a61a-585bf27153ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.495128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c347193-4179-4c9d-a61a-585bf27153ba admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.495828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c347193-4179-4c9d-a61a-585bf27153ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:28.498316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c347193-4179-4c9d-a61a-585bf27153ba admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:28.515690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dd08c0fa-2c07-4bac-8de4-df4ab7e8eca9 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.516983 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dd08c0fa-2c07-4bac-8de4-df4ab7e8eca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.540320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 777/1544] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:04:28.549178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 768/1545] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/users => generated 301 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:28.552464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead3b75-a774-41d5-9668-dcf149fe7fb4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.552763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead3b75-a774-41d5-9668-dcf149fe7fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.553052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead3b75-a774-41d5-9668-dcf149fe7fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.555084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c347193-4179-4c9d-a61a-585bf27153ba admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:28.571950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c347193-4179-4c9d-a61a-585bf27153ba admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:28.581780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233614cf-69aa-4d4f-9233-a82b929dbadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:28.582809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233614cf-69aa-4d4f-9233-a82b929dbadd admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.583325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233614cf-69aa-4d4f-9233-a82b929dbadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.583815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233614cf-69aa-4d4f-9233-a82b929dbadd admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.584781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233614cf-69aa-4d4f-9233-a82b929dbadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:28.586718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233614cf-69aa-4d4f-9233-a82b929dbadd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:28.592080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1ead3b75-a774-41d5-9668-dcf149fe7fb4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.593864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1ead3b75-a774-41d5-9668-dcf149fe7fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.608300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 769/1546] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:04:28.610565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192eca-b275-4dae-831e-cf801f45b2da None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.611063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192eca-b275-4dae-831e-cf801f45b2da None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.611532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4192eca-b275-4dae-831e-cf801f45b2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.617253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 778/1547] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:28.617253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead3b75-a774-41d5-9668-dcf149fe7fb4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:28.634494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead3b75-a774-41d5-9668-dcf149fe7fb4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:28.643452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c4192eca-b275-4dae-831e-cf801f45b2da None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.645030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c4192eca-b275-4dae-831e-cf801f45b2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.650066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dd4a6-bdc8-4577-92cc-aa6953897f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:28.650066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dd4a6-bdc8-4577-92cc-aa6953897f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.650066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dd4a6-bdc8-4577-92cc-aa6953897f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.650066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dd4a6-bdc8-4577-92cc-aa6953897f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.650493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39dd4a6-bdc8-4577-92cc-aa6953897f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:28.651106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39dd4a6-bdc8-4577-92cc-aa6953897f67 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:28.666218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 770/1548] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:28.667361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4192eca-b275-4dae-831e-cf801f45b2da None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:28.676389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 779/1549] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:04:28.677726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba101c9-1cd7-4cbc-8e67-cfc2abc7bce4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.678022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba101c9-1cd7-4cbc-8e67-cfc2abc7bce4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.678336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba101c9-1cd7-4cbc-8e67-cfc2abc7bce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.684686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4192eca-b275-4dae-831e-cf801f45b2da None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:28.695403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896abf6-c425-4fc6-a76b-f9084d59e0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:28.696363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896abf6-c425-4fc6-a76b-f9084d59e0c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.696850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896abf6-c425-4fc6-a76b-f9084d59e0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.697307 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0896abf6-c425-4fc6-a76b-f9084d59e0c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.698385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896abf6-c425-4fc6-a76b-f9084d59e0c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:04:28.702117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896abf6-c425-4fc6-a76b-f9084d59e0c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:28.702117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0896abf6-c425-4fc6-a76b-f9084d59e0c0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:28.714752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0ba101c9-1cd7-4cbc-8e67-cfc2abc7bce4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.716426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0ba101c9-1cd7-4cbc-8e67-cfc2abc7bce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.723553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 771/1550] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:04:28.724411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896abf6-c425-4fc6-a76b-f9084d59e0c0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:28.736908 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 780/1551] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:28.738600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bc66f-8aa9-4140-a8e1-cd29b2d7ea27 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.739129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bc66f-8aa9-4140-a8e1-cd29b2d7ea27 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.739650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bc66f-8aa9-4140-a8e1-cd29b2d7ea27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.746976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0896abf6-c425-4fc6-a76b-f9084d59e0c0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:28.761340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55988acc-3f76-40de-810a-d51f54bc170d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:28.762280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55988acc-3f76-40de-810a-d51f54bc170d admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.762564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55988acc-3f76-40de-810a-d51f54bc170d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.762838 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55988acc-3f76-40de-810a-d51f54bc170d admin admin] PATH_INFO: `/v3/projects/fa297495bdbe4178963461204bfbb3ff/users/96a625c49f144cb29639f4842e42402f/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.775108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ed2bc66f-8aa9-4140-a8e1-cd29b2d7ea27 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.776325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ed2bc66f-8aa9-4140-a8e1-cd29b2d7ea27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.786132 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55988acc-3f76-40de-810a-d51f54bc170d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa297495bdbe4178963461204bfbb3ff, user_id=96a625c49f144cb29639f4842e42402f, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:28.787977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-55988acc-3f76-40de-810a-d51f54bc170d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:04:28.787977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 26 03:04:28.787977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: Jun 26 03:04:28.792428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 781/1552] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:28.793489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bc66f-8aa9-4140-a8e1-cd29b2d7ea27 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:28.793871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55988acc-3f76-40de-810a-d51f54bc170d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:28.808677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bc66f-8aa9-4140-a8e1-cd29b2d7ea27 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:28.814314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260a346c-ed21-45c4-b609-a631dad2a1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:28.814988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a346c-ed21-45c4-b609-a631dad2a1be admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.815277 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a346c-ed21-45c4-b609-a631dad2a1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.815720 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260a346c-ed21-45c4-b609-a631dad2a1be admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.816463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a346c-ed21-45c4-b609-a631dad2a1be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:28.818609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 772/1553] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:28 2022] PUT /identity/v3/projects/fa297495bdbe4178963461204bfbb3ff/users/96a625c49f144cb29639f4842e42402f/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:28.818749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260a346c-ed21-45c4-b609-a631dad2a1be admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:28.819582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc345676-e93d-4920-9a74-54f3d1338a72 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.819871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc345676-e93d-4920-9a74-54f3d1338a72 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.820147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc345676-e93d-4920-9a74-54f3d1338a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.838965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cc345676-e93d-4920-9a74-54f3d1338a72 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.840279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cc345676-e93d-4920-9a74-54f3d1338a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.846680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 782/1554] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:28.851613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b581e-e050-4772-a6f0-3b9521ecf38b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.851909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b581e-e050-4772-a6f0-3b9521ecf38b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.852191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b581e-e050-4772-a6f0-3b9521ecf38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.873655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-985b581e-e050-4772-a6f0-3b9521ecf38b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.875470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-985b581e-e050-4772-a6f0-3b9521ecf38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.908727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 773/1555] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:28.909653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc345676-e93d-4920-9a74-54f3d1338a72 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:28.919200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc345676-e93d-4920-9a74-54f3d1338a72 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:28.931399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:28.939875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 783/1556] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:28.941491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc8a44-395c-4a7a-b9ba-7fd03b01715b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.941786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc8a44-395c-4a7a-b9ba-7fd03b01715b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.942087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc8a44-395c-4a7a-b9ba-7fd03b01715b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.959821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-21bc8a44-395c-4a7a-b9ba-7fd03b01715b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.961038 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-21bc8a44-395c-4a7a-b9ba-7fd03b01715b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.968945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 784/1557] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:28.970270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0935521-bc47-4d0b-9231-f216151ec0a0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.970560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0935521-bc47-4d0b-9231-f216151ec0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.970836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0935521-bc47-4d0b-9231-f216151ec0a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.988418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e0935521-bc47-4d0b-9231-f216151ec0a0 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:28.989299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5477376-bc3e-48ac-ba81-88fed0736c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:28.989603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e0935521-bc47-4d0b-9231-f216151ec0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:28.989971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477376-bc3e-48ac-ba81-88fed0736c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:28.990290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477376-bc3e-48ac-ba81-88fed0736c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:28.990573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5477376-bc3e-48ac-ba81-88fed0736c46 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:28.991310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477376-bc3e-48ac-ba81-88fed0736c46 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:04:28.991628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477376-bc3e-48ac-ba81-88fed0736c46 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:28.993238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5477376-bc3e-48ac-ba81-88fed0736c46 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:28.997360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 785/1558] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:29.001902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ff2a-5a86-42c2-b7ae-6e34b01975fa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.002960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ff2a-5a86-42c2-b7ae-6e34b01975fa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.003248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e5ff2a-5a86-42c2-b7ae-6e34b01975fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.005651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 774/1559] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:04:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:04:29.007207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb2857-bee2-4ab1-96a8-2a8606d65595 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.007514 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb2857-bee2-4ab1-96a8-2a8606d65595 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.007793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fb2857-bee2-4ab1-96a8-2a8606d65595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.024403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c6fb2857-bee2-4ab1-96a8-2a8606d65595 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:29.025631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c6fb2857-bee2-4ab1-96a8-2a8606d65595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:29.027505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-79e5ff2a-5a86-42c2-b7ae-6e34b01975fa None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:29.028664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-79e5ff2a-5a86-42c2-b7ae-6e34b01975fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:29.033038 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 775/1560] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:29.033768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb2857-bee2-4ab1-96a8-2a8606d65595 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.036457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 786/1561] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:29.037222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5ff2a-5a86-42c2-b7ae-6e34b01975fa None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.049213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e5ff2a-5a86-42c2-b7ae-6e34b01975fa None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.049545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fb2857-bee2-4ab1-96a8-2a8606d65595 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.054163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df21e26-0f98-49d7-982a-83cbc3d3e2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.054615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c589bd-8154-4292-aa8a-e0e584d5c074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.054825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21e26-0f98-49d7-982a-83cbc3d3e2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.055182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21e26-0f98-49d7-982a-83cbc3d3e2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.055306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c589bd-8154-4292-aa8a-e0e584d5c074 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.055463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df21e26-0f98-49d7-982a-83cbc3d3e2d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.055593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c589bd-8154-4292-aa8a-e0e584d5c074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.055868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c589bd-8154-4292-aa8a-e0e584d5c074 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.056192 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df21e26-0f98-49d7-982a-83cbc3d3e2d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:04:29.056501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df21e26-0f98-49d7-982a-83cbc3d3e2d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.056605 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c589bd-8154-4292-aa8a-e0e584d5c074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.057510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-1df21e26-0f98-49d7-982a-83cbc3d3e2d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:04:29.057510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 26 03:04:29.057510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: Jun 26 03:04:29.058432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df21e26-0f98-49d7-982a-83cbc3d3e2d1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.058525 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c589bd-8154-4292-aa8a-e0e584d5c074 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.079222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 787/1562] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:04:29.079222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e4c61-108b-4b45-a85e-49797029db42 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.079222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e4c61-108b-4b45-a85e-49797029db42 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.079222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985e4c61-108b-4b45-a85e-49797029db42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.082765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 776/1563] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/users => generated 294 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:29.082765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25367a27-0d7c-41bc-ab7b-da906a77c5f4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.082765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25367a27-0d7c-41bc-ab7b-da906a77c5f4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.082765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25367a27-0d7c-41bc-ab7b-da906a77c5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.092634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-985e4c61-108b-4b45-a85e-49797029db42 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:29.093873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-985e4c61-108b-4b45-a85e-49797029db42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:29.099334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-25367a27-0d7c-41bc-ab7b-da906a77c5f4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:29.100506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-25367a27-0d7c-41bc-ab7b-da906a77c5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:29.101992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 788/1564] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:29.103220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a77e8-9485-4bbd-9c3d-43a65093dc57 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.103577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a77e8-9485-4bbd-9c3d-43a65093dc57 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.103788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5a77e8-9485-4bbd-9c3d-43a65093dc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.109132 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 777/1565] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:29.110092 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25367a27-0d7c-41bc-ab7b-da906a77c5f4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.119352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25367a27-0d7c-41bc-ab7b-da906a77c5f4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.122282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ae5a77e8-9485-4bbd-9c3d-43a65093dc57 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:29.123474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ae5a77e8-9485-4bbd-9c3d-43a65093dc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:29.123880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082693a-9e6c-48d2-b4f2-4e09cfd8d0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.125202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082693a-9e6c-48d2-b4f2-4e09cfd8d0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.125492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082693a-9e6c-48d2-b4f2-4e09cfd8d0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.125769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082693a-9e6c-48d2-b4f2-4e09cfd8d0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.126626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082693a-9e6c-48d2-b4f2-4e09cfd8d0f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.127889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-1082693a-9e6c-48d2-b4f2-4e09cfd8d0f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:04:29.127889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 26 03:04:29.127889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: Jun 26 03:04:29.129076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082693a-9e6c-48d2-b4f2-4e09cfd8d0f8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.132051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 789/1566] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:29.132817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a77e8-9485-4bbd-9c3d-43a65093dc57 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.159756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 778/1567] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:29.159756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea8327-9c4a-42b1-8a22-4cef7030785f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.159756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea8327-9c4a-42b1-8a22-4cef7030785f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.159756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea8327-9c4a-42b1-8a22-4cef7030785f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.165882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:29.177543 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-12ea8327-9c4a-42b1-8a22-4cef7030785f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:29.178743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-12ea8327-9c4a-42b1-8a22-4cef7030785f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:29.194567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5a77e8-9485-4bbd-9c3d-43a65093dc57 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.236944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 779/1568] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:29.237760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea8327-9c4a-42b1-8a22-4cef7030785f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.243555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f128c-b9a4-4913-a199-63809accbad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.244332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f128c-b9a4-4913-a199-63809accbad6 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.244710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f128c-b9a4-4913-a199-63809accbad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.245088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328f128c-b9a4-4913-a199-63809accbad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.247281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea8327-9c4a-42b1-8a22-4cef7030785f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.252900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfa64c-a899-411d-8227-3e0ca95d9b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.253640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfa64c-a899-411d-8227-3e0ca95d9b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.253928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfa64c-a899-411d-8227-3e0ca95d9b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.254327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adfa64c-a899-411d-8227-3e0ca95d9b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.255074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfa64c-a899-411d-8227-3e0ca95d9b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.255367 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f128c-b9a4-4913-a199-63809accbad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.256780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328f128c-b9a4-4913-a199-63809accbad6 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.257016 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adfa64c-a899-411d-8227-3e0ca95d9b50 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.266308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 790/1569] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:29.266972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f128c-b9a4-4913-a199-63809accbad6 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.276629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328f128c-b9a4-4913-a199-63809accbad6 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.281889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a43d-9c25-4c3b-b7e1-c532ced9341f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.282638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a43d-9c25-4c3b-b7e1-c532ced9341f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.282925 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a43d-9c25-4c3b-b7e1-c532ced9341f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.283206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf6a43d-9c25-4c3b-b7e1-c532ced9341f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.283928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6a43d-9c25-4c3b-b7e1-c532ced9341f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.285443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 780/1570] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:29.285671 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf6a43d-9c25-4c3b-b7e1-c532ced9341f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.286793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfa64c-a899-411d-8227-3e0ca95d9b50 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.298135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 791/1571] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:29.301852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a43d-9c25-4c3b-b7e1-c532ced9341f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.311962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf6a43d-9c25-4c3b-b7e1-c532ced9341f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.330714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:29.365416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adfa64c-a899-411d-8227-3e0ca95d9b50 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.371304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c735c0-c4d9-422a-9d4a-847fff46d2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.371979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c735c0-c4d9-422a-9d4a-847fff46d2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.372516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c735c0-c4d9-422a-9d4a-847fff46d2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.372811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c735c0-c4d9-422a-9d4a-847fff46d2d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.373666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c735c0-c4d9-422a-9d4a-847fff46d2d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:04:29.374022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c735c0-c4d9-422a-9d4a-847fff46d2d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.375776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c735c0-c4d9-422a-9d4a-847fff46d2d1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.383837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36ecc3-3d9d-40a9-bc05-4e7db0cf9114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.384689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36ecc3-3d9d-40a9-bc05-4e7db0cf9114 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.385356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36ecc3-3d9d-40a9-bc05-4e7db0cf9114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.385695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d36ecc3-3d9d-40a9-bc05-4e7db0cf9114 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.386822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36ecc3-3d9d-40a9-bc05-4e7db0cf9114 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:04:29.387139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36ecc3-3d9d-40a9-bc05-4e7db0cf9114 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.395339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 781/1572] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:04:29.395504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d36ecc3-3d9d-40a9-bc05-4e7db0cf9114 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.395588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c735c0-c4d9-422a-9d4a-847fff46d2d1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.399918 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c735c0-c4d9-422a-9d4a-847fff46d2d1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.406315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96d8b3-4e63-4b5a-b830-7102f7112c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.407058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96d8b3-4e63-4b5a-b830-7102f7112c1d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.407411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96d8b3-4e63-4b5a-b830-7102f7112c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.407896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96d8b3-4e63-4b5a-b830-7102f7112c1d admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.408693 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96d8b3-4e63-4b5a-b830-7102f7112c1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:04:29.409101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96d8b3-4e63-4b5a-b830-7102f7112c1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.410094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 792/1573] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:04:29.410751 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36ecc3-3d9d-40a9-bc05-4e7db0cf9114 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.415425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96d8b3-4e63-4b5a-b830-7102f7112c1d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.422637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d36ecc3-3d9d-40a9-bc05-4e7db0cf9114 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.429515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc14f8-209f-4b40-80a6-d3205babd110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.430285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc14f8-209f-4b40-80a6-d3205babd110 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.430569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc14f8-209f-4b40-80a6-d3205babd110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.430847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfc14f8-209f-4b40-80a6-d3205babd110 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.431542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc14f8-209f-4b40-80a6-d3205babd110 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.432558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-cdfc14f8-209f-4b40-80a6-d3205babd110 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:04:29.432558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 26 03:04:29.432558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: Jun 26 03:04:29.433480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 782/1574] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:04:29.433782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfc14f8-209f-4b40-80a6-d3205babd110 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.439851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96d8b3-4e63-4b5a-b830-7102f7112c1d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.455811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96d8b3-4e63-4b5a-b830-7102f7112c1d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.462345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c654d-6417-4724-90b0-5b137136ebad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.463318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c654d-6417-4724-90b0-5b137136ebad admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.463529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c654d-6417-4724-90b0-5b137136ebad admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.463807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c654d-6417-4724-90b0-5b137136ebad admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.465717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 793/1575] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/users => generated 287 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:29.466207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c654d-6417-4724-90b0-5b137136ebad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:04:29.466643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c654d-6417-4724-90b0-5b137136ebad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.467080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc14f8-209f-4b40-80a6-d3205babd110 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.469351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c654d-6417-4724-90b0-5b137136ebad admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.480934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfc14f8-209f-4b40-80a6-d3205babd110 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.486266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 783/1576] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:04:29.486266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c654d-6417-4724-90b0-5b137136ebad admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.490103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977518a9-7a46-46f9-8bad-d1963d81984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.490837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977518a9-7a46-46f9-8bad-d1963d81984b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.491124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977518a9-7a46-46f9-8bad-d1963d81984b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.491398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977518a9-7a46-46f9-8bad-d1963d81984b admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.492087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977518a9-7a46-46f9-8bad-d1963d81984b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.494062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977518a9-7a46-46f9-8bad-d1963d81984b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.497556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c654d-6417-4724-90b0-5b137136ebad admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.505613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee8e6f-6503-4d75-9eac-2a24c1c16a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.506375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee8e6f-6503-4d75-9eac-2a24c1c16a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.506764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee8e6f-6503-4d75-9eac-2a24c1c16a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.506964 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee8e6f-6503-4d75-9eac-2a24c1c16a12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.507929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee8e6f-6503-4d75-9eac-2a24c1c16a12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.509766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee8e6f-6503-4d75-9eac-2a24c1c16a12 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.518525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 794/1577] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/users => generated 283 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:29.519469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977518a9-7a46-46f9-8bad-d1963d81984b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.529204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977518a9-7a46-46f9-8bad-d1963d81984b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.530074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 784/1578] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:29.530879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee8e6f-6503-4d75-9eac-2a24c1c16a12 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.534363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628ecdf-363e-4948-b406-46fd576d3640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.535045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628ecdf-363e-4948-b406-46fd576d3640 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.535525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628ecdf-363e-4948-b406-46fd576d3640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.535895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628ecdf-363e-4948-b406-46fd576d3640 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.536799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628ecdf-363e-4948-b406-46fd576d3640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.539568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d628ecdf-363e-4948-b406-46fd576d3640 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.543914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee8e6f-6503-4d75-9eac-2a24c1c16a12 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.549038 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2421a-b4fb-4a0e-a7e7-17f3d86afe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.552005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2421a-b4fb-4a0e-a7e7-17f3d86afe00 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.552005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2421a-b4fb-4a0e-a7e7-17f3d86afe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.552217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2421a-b4fb-4a0e-a7e7-17f3d86afe00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.552912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2421a-b4fb-4a0e-a7e7-17f3d86afe00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.555472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2421a-b4fb-4a0e-a7e7-17f3d86afe00 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.581532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 795/1579] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:29.582774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628ecdf-363e-4948-b406-46fd576d3640 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.589874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 785/1580] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:29.590870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2421a-b4fb-4a0e-a7e7-17f3d86afe00 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.600817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:29.676142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2421a-b4fb-4a0e-a7e7-17f3d86afe00 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.682432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f5c47-535a-40a4-911e-537d56081411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.683306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f5c47-535a-40a4-911e-537d56081411 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.683801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f5c47-535a-40a4-911e-537d56081411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.684155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f5c47-535a-40a4-911e-537d56081411 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.685010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f5c47-535a-40a4-911e-537d56081411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.687469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f5c47-535a-40a4-911e-537d56081411 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.698218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628ecdf-363e-4948-b406-46fd576d3640 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.699746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 786/1581] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:29.703953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f5c47-535a-40a4-911e-537d56081411 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.704850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08be1d3-202c-4272-89af-050f19730a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.705815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08be1d3-202c-4272-89af-050f19730a90 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.706315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08be1d3-202c-4272-89af-050f19730a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.706787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08be1d3-202c-4272-89af-050f19730a90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.707681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08be1d3-202c-4272-89af-050f19730a90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.710797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08be1d3-202c-4272-89af-050f19730a90 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.715516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f5c47-535a-40a4-911e-537d56081411 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.722486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674661a1-1966-44c1-b1b6-f1f8157b1339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.723290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674661a1-1966-44c1-b1b6-f1f8157b1339 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.723773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674661a1-1966-44c1-b1b6-f1f8157b1339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.724073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674661a1-1966-44c1-b1b6-f1f8157b1339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.725084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674661a1-1966-44c1-b1b6-f1f8157b1339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.727248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674661a1-1966-44c1-b1b6-f1f8157b1339 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.753296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 796/1582] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/projects => generated 373 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:29.754862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08be1d3-202c-4272-89af-050f19730a90 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.774506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 787/1583] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/projects => generated 383 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:29.774506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674661a1-1966-44c1-b1b6-f1f8157b1339 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.826217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:29.841560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08be1d3-202c-4272-89af-050f19730a90 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.850393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674661a1-1966-44c1-b1b6-f1f8157b1339 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.850607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282b316-356b-4baf-b5f1-826e80a7f550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.850607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282b316-356b-4baf-b5f1-826e80a7f550 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.850607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282b316-356b-4baf-b5f1-826e80a7f550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.850607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8282b316-356b-4baf-b5f1-826e80a7f550 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.850932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282b316-356b-4baf-b5f1-826e80a7f550 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.851393 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8282b316-356b-4baf-b5f1-826e80a7f550 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.858347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792360e-6163-4aad-a8c5-27aceef22647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.859789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792360e-6163-4aad-a8c5-27aceef22647 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.860078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792360e-6163-4aad-a8c5-27aceef22647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.860343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b792360e-6163-4aad-a8c5-27aceef22647 admin admin] PATH_INFO: `/v3/projects/5dd1a7a651fa4d27a1b6488ca13effca/users/430d2efc691e4aae9f96409b081c7e96/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.866223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 797/1584] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:29.866223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282b316-356b-4baf-b5f1-826e80a7f550 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.880070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792360e-6163-4aad-a8c5-27aceef22647 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd1a7a651fa4d27a1b6488ca13effca, user_id=430d2efc691e4aae9f96409b081c7e96, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.881104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8282b316-356b-4baf-b5f1-826e80a7f550 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.883130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b792360e-6163-4aad-a8c5-27aceef22647 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.885905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abb533-07b5-4bf5-854a-229ba12539e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.886663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abb533-07b5-4bf5-854a-229ba12539e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.886919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abb533-07b5-4bf5-854a-229ba12539e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.887319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94abb533-07b5-4bf5-854a-229ba12539e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.888204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abb533-07b5-4bf5-854a-229ba12539e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.890512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94abb533-07b5-4bf5-854a-229ba12539e0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:29.897944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 788/1585] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:29 2022] PUT /identity/v3/projects/5dd1a7a651fa4d27a1b6488ca13effca/users/430d2efc691e4aae9f96409b081c7e96/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:29.898851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792360e-6163-4aad-a8c5-27aceef22647 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.920305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 798/1586] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/users => generated 288 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:29.923571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abb533-07b5-4bf5-854a-229ba12539e0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:29.936131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94abb533-07b5-4bf5-854a-229ba12539e0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.986686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b792360e-6163-4aad-a8c5-27aceef22647 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:29.992157 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636446c-d3a5-42b0-a987-0f7a1ed82f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:29.992858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636446c-d3a5-42b0-a987-0f7a1ed82f12 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:29.993225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636446c-d3a5-42b0-a987-0f7a1ed82f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:29.993412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636446c-d3a5-42b0-a987-0f7a1ed82f12 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:29.994110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636446c-d3a5-42b0-a987-0f7a1ed82f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:29.996030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636446c-d3a5-42b0-a987-0f7a1ed82f12 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.007740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9fc79-6894-4ff4-a40d-21240cb93206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.008859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9fc79-6894-4ff4-a40d-21240cb93206 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.009030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9fc79-6894-4ff4-a40d-21240cb93206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.009346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a9fc79-6894-4ff4-a40d-21240cb93206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.010193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9fc79-6894-4ff4-a40d-21240cb93206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.011880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a9fc79-6894-4ff4-a40d-21240cb93206 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.025972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 799/1587] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:30.026842 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9fc79-6894-4ff4-a40d-21240cb93206 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.036143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 789/1588] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:29 2022] POST /identity/v3/users => generated 297 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:30.037194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a9fc79-6894-4ff4-a40d-21240cb93206 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.037506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636446c-d3a5-42b0-a987-0f7a1ed82f12 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.044423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158b109-bc40-4c2d-b41c-31a72cf7310d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.045326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158b109-bc40-4c2d-b41c-31a72cf7310d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.045783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158b109-bc40-4c2d-b41c-31a72cf7310d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.045966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7158b109-bc40-4c2d-b41c-31a72cf7310d admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.047349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158b109-bc40-4c2d-b41c-31a72cf7310d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.049275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7158b109-bc40-4c2d-b41c-31a72cf7310d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.049550 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636446c-d3a5-42b0-a987-0f7a1ed82f12 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.054280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d04c158-da80-4a52-8c38-f544a324d675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.055043 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04c158-da80-4a52-8c38-f544a324d675 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.055415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04c158-da80-4a52-8c38-f544a324d675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.055768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d04c158-da80-4a52-8c38-f544a324d675 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.056536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04c158-da80-4a52-8c38-f544a324d675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.058572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d04c158-da80-4a52-8c38-f544a324d675 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.061618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:30.074568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 800/1589] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:30 2022] POST /identity/v3/users => generated 287 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:30.075984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158b109-bc40-4c2d-b41c-31a72cf7310d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.081517 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 790/1590] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:30 2022] POST /identity/v3/users => generated 292 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:30.083305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319d523-9145-405c-a58a-0773e82ac3b5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.083305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319d523-9145-405c-a58a-0773e82ac3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.083658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6319d523-9145-405c-a58a-0773e82ac3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.087218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7158b109-bc40-4c2d-b41c-31a72cf7310d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.094638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a36cdff-57f4-47ac-9feb-9434874b5038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.095581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36cdff-57f4-47ac-9feb-9434874b5038 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.096154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36cdff-57f4-47ac-9feb-9434874b5038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.096436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a36cdff-57f4-47ac-9feb-9434874b5038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.097165 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36cdff-57f4-47ac-9feb-9434874b5038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.098862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a36cdff-57f4-47ac-9feb-9434874b5038 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.102256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6319d523-9145-405c-a58a-0773e82ac3b5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:30.103550 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6319d523-9145-405c-a58a-0773e82ac3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:30.114112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 791/1591] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:30.114112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319d523-9145-405c-a58a-0773e82ac3b5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.116824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 801/1592] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:30.119094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc167-37b4-4917-98e2-e597d571afbc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.119394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc167-37b4-4917-98e2-e597d571afbc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.119669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04bc167-37b4-4917-98e2-e597d571afbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.123080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6319d523-9145-405c-a58a-0773e82ac3b5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.128098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf089f-f558-4e92-aac2-e68e6202bbf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.128855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf089f-f558-4e92-aac2-e68e6202bbf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.129225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf089f-f558-4e92-aac2-e68e6202bbf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.129514 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bf089f-f558-4e92-aac2-e68e6202bbf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.130289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf089f-f558-4e92-aac2-e68e6202bbf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.131893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bf089f-f558-4e92-aac2-e68e6202bbf6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.143185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 792/1593] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:30.144005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf089f-f558-4e92-aac2-e68e6202bbf6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.149950 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e04bc167-37b4-4917-98e2-e597d571afbc None None] MFA Rules not processed for user `96a625c49f144cb29639f4842e42402f`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:30.151632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e04bc167-37b4-4917-98e2-e597d571afbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:30.154285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bf089f-f558-4e92-aac2-e68e6202bbf6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.159873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cf6bd-9e4e-4563-8e49-452d7964825f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.160740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cf6bd-9e4e-4563-8e49-452d7964825f admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.161026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cf6bd-9e4e-4563-8e49-452d7964825f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.161308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6cf6bd-9e4e-4563-8e49-452d7964825f admin admin] PATH_INFO: `/v3/projects/5fafb0ab957a4f069ef1cfea7eaa5e0e/users/0906cefead324b768df7246459704500/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.176726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cf6bd-9e4e-4563-8e49-452d7964825f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fafb0ab957a4f069ef1cfea7eaa5e0e, user_id=0906cefead324b768df7246459704500, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.180172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6cf6bd-9e4e-4563-8e49-452d7964825f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.205139 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 793/1594] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:30 2022] PUT /identity/v3/projects/5fafb0ab957a4f069ef1cfea7eaa5e0e/users/0906cefead324b768df7246459704500/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:30.205139 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cf6bd-9e4e-4563-8e49-452d7964825f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.221633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 802/1595] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:30.222567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bc167-37b4-4917-98e2-e597d571afbc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.233415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04bc167-37b4-4917-98e2-e597d571afbc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.272044 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:30.276003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6cf6bd-9e4e-4563-8e49-452d7964825f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.282332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e2638-5781-471d-835a-2407303adb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.283006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e2638-5781-471d-835a-2407303adb91 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.283316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e2638-5781-471d-835a-2407303adb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.283593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e2638-5781-471d-835a-2407303adb91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.284355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e2638-5781-471d-835a-2407303adb91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.285991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4e2638-5781-471d-835a-2407303adb91 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.290612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fd06c-9af4-4ce5-bac2-f491dafc4d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.291282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fd06c-9af4-4ce5-bac2-f491dafc4d18 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.291565 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fd06c-9af4-4ce5-bac2-f491dafc4d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.291842 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fd06c-9af4-4ce5-bac2-f491dafc4d18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.292564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4fd06c-9af4-4ce5-bac2-f491dafc4d18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.294155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4fd06c-9af4-4ce5-bac2-f491dafc4d18 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.298081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 794/1596] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:30.298081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e2638-5781-471d-835a-2407303adb91 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.306596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e2638-5781-471d-835a-2407303adb91 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.307897 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 803/1597] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:30.308564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fd06c-9af4-4ce5-bac2-f491dafc4d18 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.314071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872015f6-1d54-4ed9-a471-8387e15125ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.314708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872015f6-1d54-4ed9-a471-8387e15125ad admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.314993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872015f6-1d54-4ed9-a471-8387e15125ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.315268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872015f6-1d54-4ed9-a471-8387e15125ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.316009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872015f6-1d54-4ed9-a471-8387e15125ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.317588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872015f6-1d54-4ed9-a471-8387e15125ad admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.318025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fd06c-9af4-4ce5-bac2-f491dafc4d18 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.322602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f6b63-c557-4c3c-b31c-5adbc26e7e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.323455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f6b63-c557-4c3c-b31c-5adbc26e7e58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.323737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f6b63-c557-4c3c-b31c-5adbc26e7e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.324014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6f6b63-c557-4c3c-b31c-5adbc26e7e58 admin admin] PATH_INFO: `/v3/projects/db758d203db946e2a9b3393e1500f970/users/a852e401620f4660ae98ecc7437b07c1/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.327732 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 795/1598] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:30.331040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872015f6-1d54-4ed9-a471-8387e15125ad admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.339226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f6b63-c557-4c3c-b31c-5adbc26e7e58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db758d203db946e2a9b3393e1500f970, user_id=a852e401620f4660ae98ecc7437b07c1, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.341892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6f6b63-c557-4c3c-b31c-5adbc26e7e58 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.357587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 804/1599] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:30 2022] PUT /identity/v3/projects/db758d203db946e2a9b3393e1500f970/users/a852e401620f4660ae98ecc7437b07c1/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:30.358325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f6b63-c557-4c3c-b31c-5adbc26e7e58 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.386632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872015f6-1d54-4ed9-a471-8387e15125ad admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.403813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:30.421057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6f6b63-c557-4c3c-b31c-5adbc26e7e58 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.426200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da5b67-d4ca-4cc9-bd24-4eea882dc458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.426860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da5b67-d4ca-4cc9-bd24-4eea882dc458 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.427140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da5b67-d4ca-4cc9-bd24-4eea882dc458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.427468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da5b67-d4ca-4cc9-bd24-4eea882dc458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.428189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da5b67-d4ca-4cc9-bd24-4eea882dc458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.429900 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da5b67-d4ca-4cc9-bd24-4eea882dc458 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.435360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cdd28-3f2f-411e-8240-490fa53a28f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.436002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cdd28-3f2f-411e-8240-490fa53a28f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.436289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cdd28-3f2f-411e-8240-490fa53a28f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.436567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cdd28-3f2f-411e-8240-490fa53a28f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.440108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 805/1600] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:30.444473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a176b91-42f4-45d4-90b5-7465d5ecef02 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.444785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a176b91-42f4-45d4-90b5-7465d5ecef02 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.445049 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a176b91-42f4-45d4-90b5-7465d5ecef02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.446383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cdd28-3f2f-411e-8240-490fa53a28f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.447875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3cdd28-3f2f-411e-8240-490fa53a28f8 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.459677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 796/1601] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:30.460236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cdd28-3f2f-411e-8240-490fa53a28f8 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.469536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cdd28-3f2f-411e-8240-490fa53a28f8 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.471956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2a176b91-42f4-45d4-90b5-7465d5ecef02 None None] MFA Rules not processed for user `0906cefead324b768df7246459704500`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:30.473161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2a176b91-42f4-45d4-90b5-7465d5ecef02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:30.474263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e8a07-e75c-45a4-9db9-1eb6b225162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.474905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e8a07-e75c-45a4-9db9-1eb6b225162c admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.475188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e8a07-e75c-45a4-9db9-1eb6b225162c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.475463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e8a07-e75c-45a4-9db9-1eb6b225162c admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.476149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e8a07-e75c-45a4-9db9-1eb6b225162c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.478012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e8a07-e75c-45a4-9db9-1eb6b225162c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.510145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 797/1602] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:30 2022] POST /identity/v3/projects => generated 403 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:30.511037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e8a07-e75c-45a4-9db9-1eb6b225162c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.547832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 806/1603] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:30.548674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a176b91-42f4-45d4-90b5-7465d5ecef02 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.559834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a176b91-42f4-45d4-90b5-7465d5ecef02 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.585859 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e8a07-e75c-45a4-9db9-1eb6b225162c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.592889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c588fc3-6820-4ae2-bee6-02cd52b91910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.593666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c588fc3-6820-4ae2-bee6-02cd52b91910 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.593946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c588fc3-6820-4ae2-bee6-02cd52b91910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.594258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c588fc3-6820-4ae2-bee6-02cd52b91910 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.594985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c588fc3-6820-4ae2-bee6-02cd52b91910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.597138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c588fc3-6820-4ae2-bee6-02cd52b91910 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.613056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 798/1604] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:30.613056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c588fc3-6820-4ae2-bee6-02cd52b91910 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.622266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c588fc3-6820-4ae2-bee6-02cd52b91910 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.629734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc70c6-1d43-4483-bd12-ca829c072536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.629989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cc7bb-0656-477c-a96f-967ad12aa361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.630881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cc7bb-0656-477c-a96f-967ad12aa361 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.631134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc70c6-1d43-4483-bd12-ca829c072536 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.631420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc70c6-1d43-4483-bd12-ca829c072536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.631681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cc7bb-0656-477c-a96f-967ad12aa361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.631806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc70c6-1d43-4483-bd12-ca829c072536 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.632225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889cc7bb-0656-477c-a96f-967ad12aa361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.632415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc70c6-1d43-4483-bd12-ca829c072536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.633409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cc7bb-0656-477c-a96f-967ad12aa361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.634856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc70c6-1d43-4483-bd12-ca829c072536 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.635539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889cc7bb-0656-477c-a96f-967ad12aa361 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.653808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 799/1605] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:30.653808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62822b75-80c2-49ff-b5b4-c7ea274824c7 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.653808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62822b75-80c2-49ff-b5b4-c7ea274824c7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.653808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62822b75-80c2-49ff-b5b4-c7ea274824c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.655174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 807/1606] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:30.656135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cc7bb-0656-477c-a96f-967ad12aa361 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.659844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:30.666470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889cc7bb-0656-477c-a96f-967ad12aa361 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.672339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec1e3-8302-48b5-9bb3-43596b166895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.673227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec1e3-8302-48b5-9bb3-43596b166895 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.673511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec1e3-8302-48b5-9bb3-43596b166895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.673805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ec1e3-8302-48b5-9bb3-43596b166895 admin admin] PATH_INFO: `/v3/projects/23078d86cd394c749328fb9dd7992919/users/99e31454d9aa47f38f09f016399f9b86/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.680949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-62822b75-80c2-49ff-b5b4-c7ea274824c7 None None] MFA Rules not processed for user `a852e401620f4660ae98ecc7437b07c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:30.682828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-62822b75-80c2-49ff-b5b4-c7ea274824c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:30.692230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ec1e3-8302-48b5-9bb3-43596b166895 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23078d86cd394c749328fb9dd7992919, user_id=99e31454d9aa47f38f09f016399f9b86, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.694920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ec1e3-8302-48b5-9bb3-43596b166895 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.710881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 808/1607] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:30 2022] PUT /identity/v3/projects/23078d86cd394c749328fb9dd7992919/users/99e31454d9aa47f38f09f016399f9b86/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:30.711935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec1e3-8302-48b5-9bb3-43596b166895 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.758676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 800/1608] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:30.758676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62822b75-80c2-49ff-b5b4-c7ea274824c7 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.770466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62822b75-80c2-49ff-b5b4-c7ea274824c7 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.775230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:30.832412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ec1e3-8302-48b5-9bb3-43596b166895 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.839012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb242195-81bc-4891-b342-ac7100a63231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.840112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb242195-81bc-4891-b342-ac7100a63231 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.840318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb242195-81bc-4891-b342-ac7100a63231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.840999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb242195-81bc-4891-b342-ac7100a63231 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.841396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb242195-81bc-4891-b342-ac7100a63231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.843392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb242195-81bc-4891-b342-ac7100a63231 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.846593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c537-f66a-4ab7-85d0-beca209038bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.847495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c537-f66a-4ab7-85d0-beca209038bc admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.847950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c537-f66a-4ab7-85d0-beca209038bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.848515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb9c537-f66a-4ab7-85d0-beca209038bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.849425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9c537-f66a-4ab7-85d0-beca209038bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.851949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb9c537-f66a-4ab7-85d0-beca209038bc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.875512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 809/1609] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:30 2022] POST /identity/v3/users => generated 308 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:30.875512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb242195-81bc-4891-b342-ac7100a63231 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.885240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb242195-81bc-4891-b342-ac7100a63231 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:30.892095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21b9b3-65de-4c51-9f6f-8d7f3bdbc072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:30.893014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21b9b3-65de-4c51-9f6f-8d7f3bdbc072 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:30.893363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21b9b3-65de-4c51-9f6f-8d7f3bdbc072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:30.893693 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 801/1610] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:30 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:30.894791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c537-f66a-4ab7-85d0-beca209038bc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.897042 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce21b9b3-65de-4c51-9f6f-8d7f3bdbc072 admin admin] PATH_INFO: `/v3/projects/aa1dff9e9be549308ec0ad3a13810062/users/a1313e0937b842059f1e01480c98351c/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:30.917200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21b9b3-65de-4c51-9f6f-8d7f3bdbc072 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa1dff9e9be549308ec0ad3a13810062, user_id=a1313e0937b842059f1e01480c98351c, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:30.920109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce21b9b3-65de-4c51-9f6f-8d7f3bdbc072 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:30.938575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 810/1611] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:30 2022] PUT /identity/v3/projects/aa1dff9e9be549308ec0ad3a13810062/users/a1313e0937b842059f1e01480c98351c/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:30.939520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21b9b3-65de-4c51-9f6f-8d7f3bdbc072 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:30.976993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb9c537-f66a-4ab7-85d0-beca209038bc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.009092 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:31.015329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce21b9b3-65de-4c51-9f6f-8d7f3bdbc072 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.019988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3063388-69fa-4067-9b8d-48f32456b53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.020959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3063388-69fa-4067-9b8d-48f32456b53e admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.021171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3063388-69fa-4067-9b8d-48f32456b53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.021449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3063388-69fa-4067-9b8d-48f32456b53e admin admin] PATH_INFO: `/v3/projects/ea86edd65ecb4972ba8bb33b34387432/users/95793ccb15844069890363d983f1680d/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.036775 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3063388-69fa-4067-9b8d-48f32456b53e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea86edd65ecb4972ba8bb33b34387432, user_id=95793ccb15844069890363d983f1680d, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.039512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3063388-69fa-4067-9b8d-48f32456b53e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.049198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbd3e7-4ec2-48ce-8d5e-bfb14585f548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.050154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbd3e7-4ec2-48ce-8d5e-bfb14585f548 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.050671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbd3e7-4ec2-48ce-8d5e-bfb14585f548 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.050796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fbd3e7-4ec2-48ce-8d5e-bfb14585f548 admin admin] PATH_INFO: `/v3/projects/1850f9f425294a82add8a09690dc1df0/users/3b7a90ca5e374fc882b644e34e0a3015/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.056575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 811/1612] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:30 2022] PUT /identity/v3/projects/ea86edd65ecb4972ba8bb33b34387432/users/95793ccb15844069890363d983f1680d/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:31.058488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2a0bf-291d-4aca-9d27-424689659c0d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.058847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2a0bf-291d-4aca-9d27-424689659c0d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.059212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2a0bf-291d-4aca-9d27-424689659c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.068397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbd3e7-4ec2-48ce-8d5e-bfb14585f548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1850f9f425294a82add8a09690dc1df0, user_id=3b7a90ca5e374fc882b644e34e0a3015, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.071261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fbd3e7-4ec2-48ce-8d5e-bfb14585f548 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.081715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7eb2a0bf-291d-4aca-9d27-424689659c0d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:31.083131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7eb2a0bf-291d-4aca-9d27-424689659c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:31.085366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 802/1613] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:30 2022] PUT /identity/v3/projects/1850f9f425294a82add8a09690dc1df0/users/3b7a90ca5e374fc882b644e34e0a3015/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:31.089646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbd3e7-4ec2-48ce-8d5e-bfb14585f548 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.161121 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fbd3e7-4ec2-48ce-8d5e-bfb14585f548 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.168616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 812/1614] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:31.169555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2a0bf-291d-4aca-9d27-424689659c0d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.176175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1c880-e62e-4ee2-8c18-6a938627de46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.177042 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1c880-e62e-4ee2-8c18-6a938627de46 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.177345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1c880-e62e-4ee2-8c18-6a938627de46 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.177623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1c880-e62e-4ee2-8c18-6a938627de46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.183541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2a0bf-291d-4aca-9d27-424689659c0d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.189743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf883312-b111-4fe5-b2db-3eec7d4c6994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.190742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf883312-b111-4fe5-b2db-3eec7d4c6994 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.191052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf883312-b111-4fe5-b2db-3eec7d4c6994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.191476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf883312-b111-4fe5-b2db-3eec7d4c6994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.192226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf883312-b111-4fe5-b2db-3eec7d4c6994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.194988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf883312-b111-4fe5-b2db-3eec7d4c6994 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.209828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 813/1615] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:31.211014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf883312-b111-4fe5-b2db-3eec7d4c6994 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.226297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf883312-b111-4fe5-b2db-3eec7d4c6994 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.234902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed314b1-1035-4b7e-ac62-17b53773bd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.235529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed314b1-1035-4b7e-ac62-17b53773bd76 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.236362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed314b1-1035-4b7e-ac62-17b53773bd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.236649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed314b1-1035-4b7e-ac62-17b53773bd76 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.237348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed314b1-1035-4b7e-ac62-17b53773bd76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.239932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed314b1-1035-4b7e-ac62-17b53773bd76 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.263990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1c880-e62e-4ee2-8c18-6a938627de46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.266595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a1c880-e62e-4ee2-8c18-6a938627de46 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.272803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 814/1616] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:31 2022] POST /identity/v3/users => generated 292 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:31.272803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acc0a2-b75b-4303-8b49-1b2f9d568620 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.273060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acc0a2-b75b-4303-8b49-1b2f9d568620 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.273418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acc0a2-b75b-4303-8b49-1b2f9d568620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.281423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:31.291804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 803/1617] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:31.297608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc31873-6a77-4214-a928-bd0bb505087c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.297801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-03acc0a2-b75b-4303-8b49-1b2f9d568620 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:31.298967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-03acc0a2-b75b-4303-8b49-1b2f9d568620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:31.300383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc31873-6a77-4214-a928-bd0bb505087c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.300827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc31873-6a77-4214-a928-bd0bb505087c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.308663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 815/1618] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:31.309960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7d31c-2f96-4741-aaf8-1cd3601356f9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.310270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7d31c-2f96-4741-aaf8-1cd3601356f9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.310548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d7d31c-2f96-4741-aaf8-1cd3601356f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.338194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a5d7d31c-2f96-4741-aaf8-1cd3601356f9 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:31.339413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a5d7d31c-2f96-4741-aaf8-1cd3601356f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:31.341011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5bc31873-6a77-4214-a928-bd0bb505087c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:31.342558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5bc31873-6a77-4214-a928-bd0bb505087c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:31.349515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 816/1619] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:31.350534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7d31c-2f96-4741-aaf8-1cd3601356f9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.365765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 804/1620] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:31.366919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc31873-6a77-4214-a928-bd0bb505087c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.391081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc31873-6a77-4214-a928-bd0bb505087c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.403898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e80de-9730-45de-9fee-b8748ee76dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.405506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e80de-9730-45de-9fee-b8748ee76dde admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.406595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e80de-9730-45de-9fee-b8748ee76dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.407095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e80de-9730-45de-9fee-b8748ee76dde admin admin] PATH_INFO: `/v3/projects/ba9d39deeb954ce5928a458771b455ca/users/73c132e764b54f8db65a6756d106bca5/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.428930 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e80de-9730-45de-9fee-b8748ee76dde admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9d39deeb954ce5928a458771b455ca, user_id=73c132e764b54f8db65a6756d106bca5, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.429185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d7d31c-2f96-4741-aaf8-1cd3601356f9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.432710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e80de-9730-45de-9fee-b8748ee76dde admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.433425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0fa7c-0aac-4f9d-91b3-3f890f1a4730 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.434585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0fa7c-0aac-4f9d-91b3-3f890f1a4730 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.435792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0fa7c-0aac-4f9d-91b3-3f890f1a4730 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.436078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0fa7c-0aac-4f9d-91b3-3f890f1a4730 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.450361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0fa7c-0aac-4f9d-91b3-3f890f1a4730 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.453060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0fa7c-0aac-4f9d-91b3-3f890f1a4730 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.456606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 805/1621] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:31 2022] PUT /identity/v3/projects/ba9d39deeb954ce5928a458771b455ca/users/73c132e764b54f8db65a6756d106bca5/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:31.458586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e80de-9730-45de-9fee-b8748ee76dde admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.464815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 817/1622] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:31.477875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0fa7c-0aac-4f9d-91b3-3f890f1a4730 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.496848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0fa7c-0aac-4f9d-91b3-3f890f1a4730 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.518386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:31.583326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e80de-9730-45de-9fee-b8748ee76dde admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.598778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d2874-9c44-4eac-b523-c564edf6a232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.599533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d2874-9c44-4eac-b523-c564edf6a232 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.599815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d2874-9c44-4eac-b523-c564edf6a232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.601209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d2874-9c44-4eac-b523-c564edf6a232 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.603626 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e0ce4-5b0c-4585-b146-b844d5fb045f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.604047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d2874-9c44-4eac-b523-c564edf6a232 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.605109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e0ce4-5b0c-4585-b146-b844d5fb045f service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.605807 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e0ce4-5b0c-4585-b146-b844d5fb045f service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.606567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9e0ce4-5b0c-4585-b146-b844d5fb045f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.607495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d2874-9c44-4eac-b523-c564edf6a232 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.625729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e0ce4-5b0c-4585-b146-b844d5fb045f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.628651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9e0ce4-5b0c-4585-b146-b844d5fb045f service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.633734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 806/1623] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:31.634508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d2874-9c44-4eac-b523-c564edf6a232 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.643722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 818/1624] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:31.644826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e0ce4-5b0c-4585-b146-b844d5fb045f service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.646785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0d9e0ce4-5b0c-4585-b146-b844d5fb045f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:31.663666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d2874-9c44-4eac-b523-c564edf6a232 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.664697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9e0ce4-5b0c-4585-b146-b844d5fb045f service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.671095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0103b3-e019-4963-bcc2-f1e371087ba5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.672598 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0103b3-e019-4963-bcc2-f1e371087ba5 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.673026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0103b3-e019-4963-bcc2-f1e371087ba5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.673316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0103b3-e019-4963-bcc2-f1e371087ba5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.673621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616803c9-73a7-4124-8359-a501298e89fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.675862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616803c9-73a7-4124-8359-a501298e89fe service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.676149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616803c9-73a7-4124-8359-a501298e89fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.677147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616803c9-73a7-4124-8359-a501298e89fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.696031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0103b3-e019-4963-bcc2-f1e371087ba5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.699469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616803c9-73a7-4124-8359-a501298e89fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.701359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616803c9-73a7-4124-8359-a501298e89fe service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.701609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0103b3-e019-4963-bcc2-f1e371087ba5 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.720108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 807/1625] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:31.721064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616803c9-73a7-4124-8359-a501298e89fe service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.729725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 819/1626] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:31.733207 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0103b3-e019-4963-bcc2-f1e371087ba5 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.733378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616803c9-73a7-4124-8359-a501298e89fe service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.741848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b747d-174a-4555-b09e-32adef156a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.742900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b747d-174a-4555-b09e-32adef156a71 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.743275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b747d-174a-4555-b09e-32adef156a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.743619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b747d-174a-4555-b09e-32adef156a71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.744631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b747d-174a-4555-b09e-32adef156a71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.746729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b747d-174a-4555-b09e-32adef156a71 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.762737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0103b3-e019-4963-bcc2-f1e371087ba5 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.769131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 808/1627] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:31.770261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b747d-174a-4555-b09e-32adef156a71 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.775961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8f7e2-c300-4494-bd24-e370a92f7c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.777375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8f7e2-c300-4494-bd24-e370a92f7c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.777661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8f7e2-c300-4494-bd24-e370a92f7c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.777935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8f7e2-c300-4494-bd24-e370a92f7c4b admin admin] PATH_INFO: `/v3/projects/7065a845fd404bbfbf11c1ad766e8c09/users/bb6338d89ee3406094136ba08161623f/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.784875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b747d-174a-4555-b09e-32adef156a71 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:31.792755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437edf44-8aeb-4c1e-877f-87ede5c122a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:31.809237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437edf44-8aeb-4c1e-877f-87ede5c122a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:31.809774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437edf44-8aeb-4c1e-877f-87ede5c122a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:31.810141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437edf44-8aeb-4c1e-877f-87ede5c122a7 admin admin] PATH_INFO: `/v3/projects/ba9d39deeb954ce5928a458771b455ca/users/73c132e764b54f8db65a6756d106bca5/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:31.815803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437edf44-8aeb-4c1e-877f-87ede5c122a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba9d39deeb954ce5928a458771b455ca, user_id=73c132e764b54f8db65a6756d106bca5, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.816283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8f7e2-c300-4494-bd24-e370a92f7c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7065a845fd404bbfbf11c1ad766e8c09, user_id=bb6338d89ee3406094136ba08161623f, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:31.820119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437edf44-8aeb-4c1e-877f-87ede5c122a7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.824652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e8f7e2-c300-4494-bd24-e370a92f7c4b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:31.840446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 809/1628] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:31 2022] PUT /identity/v3/projects/ba9d39deeb954ce5928a458771b455ca/users/73c132e764b54f8db65a6756d106bca5/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:31.860267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 820/1629] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:31 2022] PUT /identity/v3/projects/7065a845fd404bbfbf11c1ad766e8c09/users/bb6338d89ee3406094136ba08161623f/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:31.871602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437edf44-8aeb-4c1e-877f-87ede5c122a7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:31.974988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437edf44-8aeb-4c1e-877f-87ede5c122a7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:32.062845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbf1c7-7807-4433-9240-95ee5c4afa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:32.063618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbf1c7-7807-4433-9240-95ee5c4afa8f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:32.064815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbf1c7-7807-4433-9240-95ee5c4afa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:32.065098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbf1c7-7807-4433-9240-95ee5c4afa8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:32.066187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbf1c7-7807-4433-9240-95ee5c4afa8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:32.068125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbf1c7-7807-4433-9240-95ee5c4afa8f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:32.089206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 810/1630] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:32.090761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8f7e2-c300-4494-bd24-e370a92f7c4b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:32.111202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8f7e2-c300-4494-bd24-e370a92f7c4b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:32.119087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294386b3-5fc8-4266-9b04-7f4ee80bc037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:32.120132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294386b3-5fc8-4266-9b04-7f4ee80bc037 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:32.120416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294386b3-5fc8-4266-9b04-7f4ee80bc037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:32.120847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294386b3-5fc8-4266-9b04-7f4ee80bc037 admin admin] PATH_INFO: `/v3/projects/7065a845fd404bbfbf11c1ad766e8c09/users/bb6338d89ee3406094136ba08161623f/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:32.123876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294386b3-5fc8-4266-9b04-7f4ee80bc037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7065a845fd404bbfbf11c1ad766e8c09, user_id=bb6338d89ee3406094136ba08161623f, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:32.129496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294386b3-5fc8-4266-9b04-7f4ee80bc037 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:32.150088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 821/1631] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:32 2022] PUT /identity/v3/projects/7065a845fd404bbfbf11c1ad766e8c09/users/bb6338d89ee3406094136ba08161623f/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:32.158540 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d4fe-1031-4f47-b897-15c798e75f7c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:32.159138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d4fe-1031-4f47-b897-15c798e75f7c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:32.161075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f1d4fe-1031-4f47-b897-15c798e75f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:32.205245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d4f1d4fe-1031-4f47-b897-15c798e75f7c None None] MFA Rules not processed for user `bb6338d89ee3406094136ba08161623f`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:32.219865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d4f1d4fe-1031-4f47-b897-15c798e75f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:32.325232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:32.448471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 811/1632] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:32 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:32.461665 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294386b3-5fc8-4266-9b04-7f4ee80bc037 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:32.543011 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:32.566637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294386b3-5fc8-4266-9b04-7f4ee80bc037 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:32.575959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2129b91b-3094-40cd-8860-301898ae6146 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:32.576997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129b91b-3094-40cd-8860-301898ae6146 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:32.578358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129b91b-3094-40cd-8860-301898ae6146 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:32.578797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2129b91b-3094-40cd-8860-301898ae6146 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:32.593853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129b91b-3094-40cd-8860-301898ae6146 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:32.595468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2129b91b-3094-40cd-8860-301898ae6146 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:32.620248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 822/1633] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:32 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:32.760351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d4fe-1031-4f47-b897-15c798e75f7c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:32.779341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f1d4fe-1031-4f47-b897-15c798e75f7c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:32.791442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13700071-b3d6-489c-aced-d6042ecdb7c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:32.792560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13700071-b3d6-489c-aced-d6042ecdb7c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:32.793058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13700071-b3d6-489c-aced-d6042ecdb7c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:32.793529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13700071-b3d6-489c-aced-d6042ecdb7c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:32.892437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13700071-b3d6-489c-aced-d6042ecdb7c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:32.894202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13700071-b3d6-489c-aced-d6042ecdb7c3 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:32.905356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 812/1634] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:34.920504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797a56f-7dc1-4c65-94e8-29fea9fab85d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:34.922931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797a56f-7dc1-4c65-94e8-29fea9fab85d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:34.923491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6797a56f-7dc1-4c65-94e8-29fea9fab85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:34.993005 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6797a56f-7dc1-4c65-94e8-29fea9fab85d None None] MFA Rules not processed for user `73c132e764b54f8db65a6756d106bca5`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:34.995452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6797a56f-7dc1-4c65-94e8-29fea9fab85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:35.115503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 823/1635] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:34 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:35.224493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3871d7b-f471-4663-abfb-8210c106fec3 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:35.225053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3871d7b-f471-4663-abfb-8210c106fec3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:35.225053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3871d7b-f471-4663-abfb-8210c106fec3 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:35.226508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 813/1636] 10.222.0.13 () {54 vars in 961 bytes} [Sun Jun 26 03:04:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:04:35.259602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081695c-a6a2-461b-9b51-97a21e865951 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:35.259956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081695c-a6a2-461b-9b51-97a21e865951 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:35.260249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3081695c-a6a2-461b-9b51-97a21e865951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:35.288507 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3081695c-a6a2-461b-9b51-97a21e865951 None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:35.289917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3081695c-a6a2-461b-9b51-97a21e865951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:35.402272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 824/1637] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Jun 26 03:04:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:04:35.407979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a5fd6-dae4-42f6-8566-b226554d3814 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:35.409087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a5fd6-dae4-42f6-8566-b226554d3814 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:35.410102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a5fd6-dae4-42f6-8566-b226554d3814 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:35.411452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 814/1638] 10.222.0.13 () {54 vars in 912 bytes} [Sun Jun 26 03:04:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:04:35.419981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0caff4-6b6f-4521-aed0-ba6f50b79fa4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:35.422408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0caff4-6b6f-4521-aed0-ba6f50b79fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:35.423418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0caff4-6b6f-4521-aed0-ba6f50b79fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:35.453906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9b0caff4-6b6f-4521-aed0-ba6f50b79fa4 None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:35.456104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9b0caff4-6b6f-4521-aed0-ba6f50b79fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:35.468497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 825/1639] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:04:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:35.481694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a5fd6-dae4-42f6-8566-b226554d3814 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:35.502254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a5fd6-dae4-42f6-8566-b226554d3814 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:35.513903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd5866-53b3-42f7-8fec-bbeaab98c9ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:35.514605 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd5866-53b3-42f7-8fec-bbeaab98c9ac service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:35.515455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd5866-53b3-42f7-8fec-bbeaab98c9ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:35.515738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd5866-53b3-42f7-8fec-bbeaab98c9ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:35.541775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd5866-53b3-42f7-8fec-bbeaab98c9ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:35.543524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd5866-53b3-42f7-8fec-bbeaab98c9ac service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:35.582897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 815/1640] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:38.027601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877ffa9-c04a-4412-930b-1cc9d20d9972 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:38.028233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877ffa9-c04a-4412-930b-1cc9d20d9972 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:38.028461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0877ffa9-c04a-4412-930b-1cc9d20d9972 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:38.030949 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 826/1641] 10.222.0.13 () {54 vars in 961 bytes} [Sun Jun 26 03:04:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 03:04:38.044449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eeb83-368b-4ec1-be92-633c5935c9fa None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:38.045939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eeb83-368b-4ec1-be92-633c5935c9fa None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:38.046471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423eeb83-368b-4ec1-be92-633c5935c9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:38.106690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-423eeb83-368b-4ec1-be92-633c5935c9fa None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:38.109676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-423eeb83-368b-4ec1-be92-633c5935c9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:38.144740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 816/1642] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Jun 26 03:04:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:04:38.153184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8603bcd-c90d-44b3-8c2b-ec861d280c8e None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:38.153611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8603bcd-c90d-44b3-8c2b-ec861d280c8e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:38.153951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8603bcd-c90d-44b3-8c2b-ec861d280c8e None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:38.161885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bff64-5eb2-4e33-805d-9542c8304323 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:38.162189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bff64-5eb2-4e33-805d-9542c8304323 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:38.162636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3bff64-5eb2-4e33-805d-9542c8304323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:38.162790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 827/1643] 10.222.0.13 () {54 vars in 912 bytes} [Sun Jun 26 03:04:38 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:04:38.211860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8b3bff64-5eb2-4e33-805d-9542c8304323 None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:38.213187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8b3bff64-5eb2-4e33-805d-9542c8304323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:38.224797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 817/1644] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:04:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:04:38.266685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8603bcd-c90d-44b3-8c2b-ec861d280c8e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:38.298924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8603bcd-c90d-44b3-8c2b-ec861d280c8e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:38.327976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3461485-34fb-4628-ab81-56b3f6c3bf5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:38.329025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3461485-34fb-4628-ab81-56b3f6c3bf5b service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:38.329311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3461485-34fb-4628-ab81-56b3f6c3bf5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:38.329584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3461485-34fb-4628-ab81-56b3f6c3bf5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:38.352925 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3461485-34fb-4628-ab81-56b3f6c3bf5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:38.354385 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3461485-34fb-4628-ab81-56b3f6c3bf5b service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:38.388022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 828/1645] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:04:38.490785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b57b13-d5a4-464a-9079-47f0359f7a34 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:38.491131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b57b13-d5a4-464a-9079-47f0359f7a34 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:38.491450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b57b13-d5a4-464a-9079-47f0359f7a34 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:38.493302 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 818/1646] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:04:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:04:38.595599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3df6d0-4043-42cc-b237-c5507f2f845b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:38.596652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3df6d0-4043-42cc-b237-c5507f2f845b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:38.596652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3df6d0-4043-42cc-b237-c5507f2f845b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:38.631810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-eb3df6d0-4043-42cc-b237-c5507f2f845b None None] MFA Rules not processed for user `19ce6f259c22441486bfc04811a271b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:38.637234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eb3df6d0-4043-42cc-b237-c5507f2f845b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:38.730183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:38.781159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 829/1647] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:04:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:38.823480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b57b13-d5a4-464a-9079-47f0359f7a34 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:38.854382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b57b13-d5a4-464a-9079-47f0359f7a34 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:38.874849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22114bc-9eb5-478d-afd5-4b56130468e3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:38.875536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22114bc-9eb5-478d-afd5-4b56130468e3 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:38.875806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22114bc-9eb5-478d-afd5-4b56130468e3 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:38.876077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22114bc-9eb5-478d-afd5-4b56130468e3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:38.899047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22114bc-9eb5-478d-afd5-4b56130468e3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:38.901553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22114bc-9eb5-478d-afd5-4b56130468e3 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:38.925084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 819/1648] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:41.485005 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3df6d0-4043-42cc-b237-c5507f2f845b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:41.677678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3df6d0-4043-42cc-b237-c5507f2f845b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:41.692819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:41.693810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:41.694123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:41.694413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] PATH_INFO: `/v3/users/96a625c49f144cb29639f4842e42402f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:41.695652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96a625c49f144cb29639f4842e42402f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:41.697626 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:41.817120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96a625c49f144cb29639f4842e42402f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:41.817845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96a625c49f144cb29639f4842e42402f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:41.833249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96a625c49f144cb29639f4842e42402f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:41.845429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96a625c49f144cb29639f4842e42402f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:41.846132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96a625c49f144cb29639f4842e42402f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:41.880473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 830/1649] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:04:41 2022] DELETE /identity/v3/users/96a625c49f144cb29639f4842e42402f => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:04:42.068899 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdabc28-7325-46a0-8a7f-465a125fa4b4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:42.070152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdabc28-7325-46a0-8a7f-465a125fa4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:42.070868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdabc28-7325-46a0-8a7f-465a125fa4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:42.139184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8cdabc28-7325-46a0-8a7f-465a125fa4b4 None None] MFA Rules not processed for user `430d2efc691e4aae9f96409b081c7e96`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:42.140492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8cdabc28-7325-46a0-8a7f-465a125fa4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:42.276661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 820/1650] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:42.292323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:42.295511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:42.417849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:42.457609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2aa62-3b38-46dc-b811-1cad661a9088 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:42.472650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83287d71-6d0a-4bfc-837c-ce8f6fa23501 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:42.473254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83287d71-6d0a-4bfc-837c-ce8f6fa23501 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:42.473542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83287d71-6d0a-4bfc-837c-ce8f6fa23501 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:42.473874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83287d71-6d0a-4bfc-837c-ce8f6fa23501 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:42.486314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83287d71-6d0a-4bfc-837c-ce8f6fa23501 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:42.487952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83287d71-6d0a-4bfc-837c-ce8f6fa23501 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:42.512849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 831/1651] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:42 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:43.597784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdabc28-7325-46a0-8a7f-465a125fa4b4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:43.600331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8cdabc28-7325-46a0-8a7f-465a125fa4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:43.656159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:43.844200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdabc28-7325-46a0-8a7f-465a125fa4b4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:43.875924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:43.877406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:43.877941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:43.880592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] PATH_INFO: `/v3/projects/fa297495bdbe4178963461204bfbb3ff` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:43.897089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa297495bdbe4178963461204bfbb3ff)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:43.904539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:44.028671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa297495bdbe4178963461204bfbb3ff', 'request_id': 'req-46f04bc7-a455-47dd-b0e6-dab4939fcb31'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:44.063751 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa297495bdbe4178963461204bfbb3ff', 'request_id': 'req-46f04bc7-a455-47dd-b0e6-dab4939fcb31'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:44.072053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa297495bdbe4178963461204bfbb3ff', 'request_id': 'req-46f04bc7-a455-47dd-b0e6-dab4939fcb31'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:44.094508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] The token cache is being invalidate because project fa297495bdbe4178963461204bfbb3ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:04:44.096027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:44.097215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-46f04bc7-a455-47dd-b0e6-dab4939fcb31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:44.100790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 821/1652] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:04:43 2022] DELETE /identity/v3/projects/fa297495bdbe4178963461204bfbb3ff => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:44.105258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83287d71-6d0a-4bfc-837c-ce8f6fa23501 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:44.121418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83287d71-6d0a-4bfc-837c-ce8f6fa23501 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:44.322993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:44.323912 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:44.324203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:44.324478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] PATH_INFO: `/v3/users/73c132e764b54f8db65a6756d106bca5` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:44.330629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=73c132e764b54f8db65a6756d106bca5)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:44.332623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:44.433605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73c132e764b54f8db65a6756d106bca5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:44.434583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73c132e764b54f8db65a6756d106bca5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:44.458711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73c132e764b54f8db65a6756d106bca5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:44.477347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73c132e764b54f8db65a6756d106bca5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:44.478682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73c132e764b54f8db65a6756d106bca5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:44.508781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 832/1653] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:04:44 2022] DELETE /identity/v3/users/73c132e764b54f8db65a6756d106bca5 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:04:46.102214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add814ec-f761-443d-9bdc-d9f3d20bf392 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:46.103213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add814ec-f761-443d-9bdc-d9f3d20bf392 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:46.103635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add814ec-f761-443d-9bdc-d9f3d20bf392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:46.163717 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-add814ec-f761-443d-9bdc-d9f3d20bf392 None None] MFA Rules not processed for user `95793ccb15844069890363d983f1680d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:46.165017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-add814ec-f761-443d-9bdc-d9f3d20bf392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:46.412555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 822/1654] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:46.425470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:46.427045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:46.576837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:46.579102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add814ec-f761-443d-9bdc-d9f3d20bf392 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:46.580105 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-add814ec-f761-443d-9bdc-d9f3d20bf392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:46.603893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb46ee85-a73e-42b7-8354-b0810706f5bc admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:46.625942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49575628-c0c6-4846-89eb-55fed6453029 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:46.626672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49575628-c0c6-4846-89eb-55fed6453029 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:46.626952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49575628-c0c6-4846-89eb-55fed6453029 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:46.627242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49575628-c0c6-4846-89eb-55fed6453029 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:46.651087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49575628-c0c6-4846-89eb-55fed6453029 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:46.656567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49575628-c0c6-4846-89eb-55fed6453029 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:46.668408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:46.698229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 833/1655] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:46.773329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add814ec-f761-443d-9bdc-d9f3d20bf392 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:46.791902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:46.793070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:46.795867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:46.796169 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] PATH_INFO: `/v3/projects/ba9d39deeb954ce5928a458771b455ca` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:46.799910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba9d39deeb954ce5928a458771b455ca)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:46.804083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:46.994754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba9d39deeb954ce5928a458771b455ca', 'request_id': 'req-0397c526-f122-4226-b158-a2336085e027'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:47.012196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba9d39deeb954ce5928a458771b455ca', 'request_id': 'req-0397c526-f122-4226-b158-a2336085e027'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:47.028708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba9d39deeb954ce5928a458771b455ca', 'request_id': 'req-0397c526-f122-4226-b158-a2336085e027'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:47.085040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] The token cache is being invalidate because project ba9d39deeb954ce5928a458771b455ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:04:47.085040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:47.094286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0397c526-f122-4226-b158-a2336085e027 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:47.099482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 823/1656] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:04:46 2022] DELETE /identity/v3/projects/ba9d39deeb954ce5928a458771b455ca => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:47.104805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d916401-b781-46aa-bbf2-bb42a16eba2b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:47.105302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d916401-b781-46aa-bbf2-bb42a16eba2b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:47.106015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d916401-b781-46aa-bbf2-bb42a16eba2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:47.156390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0d916401-b781-46aa-bbf2-bb42a16eba2b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:47.162788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0d916401-b781-46aa-bbf2-bb42a16eba2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:47.351288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 834/1657] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:47.359762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538bd1-45fd-41e5-b7f1-0182ed62085d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:47.360305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538bd1-45fd-41e5-b7f1-0182ed62085d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:47.360305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65538bd1-45fd-41e5-b7f1-0182ed62085d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:47.416622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-65538bd1-45fd-41e5-b7f1-0182ed62085d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:47.419841 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-65538bd1-45fd-41e5-b7f1-0182ed62085d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:47.456978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 824/1658] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:47.461670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d916401-b781-46aa-bbf2-bb42a16eba2b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:47.483326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d916401-b781-46aa-bbf2-bb42a16eba2b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:47.515384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf470a13-260b-4eec-8e9a-90e1bf8f874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:47.516102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf470a13-260b-4eec-8e9a-90e1bf8f874f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:47.516382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf470a13-260b-4eec-8e9a-90e1bf8f874f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:47.516657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf470a13-260b-4eec-8e9a-90e1bf8f874f admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:47.517399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf470a13-260b-4eec-8e9a-90e1bf8f874f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:04:47.517708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf470a13-260b-4eec-8e9a-90e1bf8f874f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:47.519558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf470a13-260b-4eec-8e9a-90e1bf8f874f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:47.555992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 835/1659] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:04:47.562184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538bd1-45fd-41e5-b7f1-0182ed62085d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:47.575718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65538bd1-45fd-41e5-b7f1-0182ed62085d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:47.606449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c703d-04e9-46d6-9fc0-9acfa69c58fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:47.607146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c703d-04e9-46d6-9fc0-9acfa69c58fd admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:47.607432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c703d-04e9-46d6-9fc0-9acfa69c58fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:47.607710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6c703d-04e9-46d6-9fc0-9acfa69c58fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:47.608453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c703d-04e9-46d6-9fc0-9acfa69c58fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:47.610405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6c703d-04e9-46d6-9fc0-9acfa69c58fd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:47.697129 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 825/1660] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:47 2022] POST /identity/v3/projects => generated 381 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:47.700710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf470a13-260b-4eec-8e9a-90e1bf8f874f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:47.716674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf470a13-260b-4eec-8e9a-90e1bf8f874f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:47.946900 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3be0fd-da71-49d8-a311-a497c84ba16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:47.947708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3be0fd-da71-49d8-a311-a497c84ba16c admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:47.948854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3be0fd-da71-49d8-a311-a497c84ba16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:47.949228 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3be0fd-da71-49d8-a311-a497c84ba16c admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:47.950092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3be0fd-da71-49d8-a311-a497c84ba16c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:47.963470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3be0fd-da71-49d8-a311-a497c84ba16c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:48.018358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 836/1661] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:47 2022] POST /identity/v3/users => generated 291 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:48.022804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c703d-04e9-46d6-9fc0-9acfa69c58fd admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:48.037356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6c703d-04e9-46d6-9fc0-9acfa69c58fd admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:48.058230 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e1e46-02c1-4abd-8224-c52c5a1be954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:48.058927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e1e46-02c1-4abd-8224-c52c5a1be954 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:48.059212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e1e46-02c1-4abd-8224-c52c5a1be954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:48.062235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33e1e46-02c1-4abd-8224-c52c5a1be954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:48.064470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e1e46-02c1-4abd-8224-c52c5a1be954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:48.087213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33e1e46-02c1-4abd-8224-c52c5a1be954 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:48.107342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 826/1662] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:48.114111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3be0fd-da71-49d8-a311-a497c84ba16c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:48.124903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3be0fd-da71-49d8-a311-a497c84ba16c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:48.140122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da4d1aa-a638-421a-b791-36c5c6cd5bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:48.141272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4d1aa-a638-421a-b791-36c5c6cd5bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:48.141528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4d1aa-a638-421a-b791-36c5c6cd5bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:48.141803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da4d1aa-a638-421a-b791-36c5c6cd5bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:48.145463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4d1aa-a638-421a-b791-36c5c6cd5bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:48.147241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da4d1aa-a638-421a-b791-36c5c6cd5bc5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:48.177119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 837/1663] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:48.182303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e1e46-02c1-4abd-8224-c52c5a1be954 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:48.198585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33e1e46-02c1-4abd-8224-c52c5a1be954 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:48.207881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec76e0-14f8-4146-a123-dee82e19a8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:48.209239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec76e0-14f8-4146-a123-dee82e19a8a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:48.209542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec76e0-14f8-4146-a123-dee82e19a8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:48.209836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec76e0-14f8-4146-a123-dee82e19a8a1 admin admin] PATH_INFO: `/v3/projects/b9afae2a48e34178bdb031839b1781bf/users/4137ef42899c4addb833f11d453c2243/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:48.249226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec76e0-14f8-4146-a123-dee82e19a8a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9afae2a48e34178bdb031839b1781bf, user_id=4137ef42899c4addb833f11d453c2243, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:48.252207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bec76e0-14f8-4146-a123-dee82e19a8a1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:48.298564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 827/1664] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:48 2022] PUT /identity/v3/projects/b9afae2a48e34178bdb031839b1781bf/users/4137ef42899c4addb833f11d453c2243/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:48.309873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720479-e1ac-49ac-8003-d31f5eea9bab None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:48.310196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720479-e1ac-49ac-8003-d31f5eea9bab None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:48.310742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21720479-e1ac-49ac-8003-d31f5eea9bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:48.401265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-21720479-e1ac-49ac-8003-d31f5eea9bab None None] MFA Rules not processed for user `4137ef42899c4addb833f11d453c2243`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:48.406308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-21720479-e1ac-49ac-8003-d31f5eea9bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:48.548725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:48.637022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 838/1665] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:04:48.641683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec76e0-14f8-4146-a123-dee82e19a8a1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:48.795118 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:48.807842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec76e0-14f8-4146-a123-dee82e19a8a1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:48.830869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338cbb0-cf4e-432d-9f46-a019c0710ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:48.831861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338cbb0-cf4e-432d-9f46-a019c0710ac6 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:48.833461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338cbb0-cf4e-432d-9f46-a019c0710ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:48.833808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338cbb0-cf4e-432d-9f46-a019c0710ac6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:48.834722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338cbb0-cf4e-432d-9f46-a019c0710ac6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:48.838971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338cbb0-cf4e-432d-9f46-a019c0710ac6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:48.916113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 828/1666] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:48 2022] POST /identity/v3/projects => generated 381 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:48.920445 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21720479-e1ac-49ac-8003-d31f5eea9bab None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:48.935087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21720479-e1ac-49ac-8003-d31f5eea9bab None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:49.002650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0feaf-9aa2-4881-b7e4-a910eec2f308 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:49.003282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0feaf-9aa2-4881-b7e4-a910eec2f308 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:49.003634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde0feaf-9aa2-4881-b7e4-a910eec2f308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:49.062326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-fde0feaf-9aa2-4881-b7e4-a910eec2f308 None None] MFA Rules not processed for user `99e31454d9aa47f38f09f016399f9b86`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:49.062536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fca95-b062-4072-ac5a-091cfabb3902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:49.064314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fca95-b062-4072-ac5a-091cfabb3902 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:49.064596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fca95-b062-4072-ac5a-091cfabb3902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:49.065325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fde0feaf-9aa2-4881-b7e4-a910eec2f308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:49.065493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328fca95-b062-4072-ac5a-091cfabb3902 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:49.067024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fca95-b062-4072-ac5a-091cfabb3902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:49.070987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328fca95-b062-4072-ac5a-091cfabb3902 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:49.125472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 839/1667] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:48 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:49.130465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fca95-b062-4072-ac5a-091cfabb3902 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:49.146554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328fca95-b062-4072-ac5a-091cfabb3902 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:49.154683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68716b-8ac4-46f6-9010-4e22b27b6195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:49.156451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68716b-8ac4-46f6-9010-4e22b27b6195 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:49.156880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68716b-8ac4-46f6-9010-4e22b27b6195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:49.157295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68716b-8ac4-46f6-9010-4e22b27b6195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:49.159322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68716b-8ac4-46f6-9010-4e22b27b6195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:49.161453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68716b-8ac4-46f6-9010-4e22b27b6195 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:49.177169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 840/1668] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:49.181223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68716b-8ac4-46f6-9010-4e22b27b6195 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:49.193875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68716b-8ac4-46f6-9010-4e22b27b6195 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:49.200769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1339b-58ae-469e-abe9-99a1c890bf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:49.202454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1339b-58ae-469e-abe9-99a1c890bf27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:49.202752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1339b-58ae-469e-abe9-99a1c890bf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:49.203031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1339b-58ae-469e-abe9-99a1c890bf27 admin admin] PATH_INFO: `/v3/projects/766319fd756043ec966c9d6f82565c41/users/d1c93b02e4574506ad560ba840d3575e/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:49.229448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1339b-58ae-469e-abe9-99a1c890bf27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=766319fd756043ec966c9d6f82565c41, user_id=d1c93b02e4574506ad560ba840d3575e, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:49.248290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1339b-58ae-469e-abe9-99a1c890bf27 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:49.279924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 841/1669] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:49 2022] PUT /identity/v3/projects/766319fd756043ec966c9d6f82565c41/users/d1c93b02e4574506ad560ba840d3575e/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:49.284297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1339b-58ae-469e-abe9-99a1c890bf27 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:49.334074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 829/1670] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 322 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:49.340753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556e88c-2cd8-4738-91ef-a7ee04ab16f8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:49.349773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556e88c-2cd8-4738-91ef-a7ee04ab16f8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:49.350229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556e88c-2cd8-4738-91ef-a7ee04ab16f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:49.442945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5556e88c-2cd8-4738-91ef-a7ee04ab16f8 None None] MFA Rules not processed for user `99e31454d9aa47f38f09f016399f9b86`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:49.444214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5556e88c-2cd8-4738-91ef-a7ee04ab16f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:49.464478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1339b-58ae-469e-abe9-99a1c890bf27 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:49.479221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a07b1a-10a3-40c6-b13b-ee79844da7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:49.484444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a07b1a-10a3-40c6-b13b-ee79844da7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:49.484754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a07b1a-10a3-40c6-b13b-ee79844da7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:49.485024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a07b1a-10a3-40c6-b13b-ee79844da7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:49.485791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a07b1a-10a3-40c6-b13b-ee79844da7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:49.488075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a07b1a-10a3-40c6-b13b-ee79844da7ce admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:49.517167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 842/1671] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:49.521822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a07b1a-10a3-40c6-b13b-ee79844da7ce admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:49.537575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a07b1a-10a3-40c6-b13b-ee79844da7ce admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:49.547204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3065c4-3f6f-43dc-bdd0-8682b4a6e9aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:49.548597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3065c4-3f6f-43dc-bdd0-8682b4a6e9aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:49.549325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3065c4-3f6f-43dc-bdd0-8682b4a6e9aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:49.549842 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3065c4-3f6f-43dc-bdd0-8682b4a6e9aa admin admin] PATH_INFO: `/v3/projects/766319fd756043ec966c9d6f82565c41/users/d1c93b02e4574506ad560ba840d3575e/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:49.551873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3065c4-3f6f-43dc-bdd0-8682b4a6e9aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=766319fd756043ec966c9d6f82565c41, user_id=d1c93b02e4574506ad560ba840d3575e, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:49.555129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3065c4-3f6f-43dc-bdd0-8682b4a6e9aa admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:49.575369 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 843/1672] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:49 2022] PUT /identity/v3/projects/766319fd756043ec966c9d6f82565c41/users/d1c93b02e4574506ad560ba840d3575e/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:49.588305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850fd28-492e-42ad-8107-d1ab00c22f4c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:49.588741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850fd28-492e-42ad-8107-d1ab00c22f4c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:49.589705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9850fd28-492e-42ad-8107-d1ab00c22f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:49.687631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 830/1673] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:49.697864 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556e88c-2cd8-4738-91ef-a7ee04ab16f8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:49.700077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5556e88c-2cd8-4738-91ef-a7ee04ab16f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:49.709340 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9850fd28-492e-42ad-8107-d1ab00c22f4c None None] MFA Rules not processed for user `d1c93b02e4574506ad560ba840d3575e`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:49.714409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9850fd28-492e-42ad-8107-d1ab00c22f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:49.749150 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:49.939818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556e88c-2cd8-4738-91ef-a7ee04ab16f8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:49.980087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 844/1674] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 389 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:49.989960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:50.007806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850fd28-492e-42ad-8107-d1ab00c22f4c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:50.009316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9850fd28-492e-42ad-8107-d1ab00c22f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:50.131710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9850fd28-492e-42ad-8107-d1ab00c22f4c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:50.138754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac848c9-d87c-4734-9aab-8f83ab3d1ed8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:50.139673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac848c9-d87c-4734-9aab-8f83ab3d1ed8 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:50.139965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac848c9-d87c-4734-9aab-8f83ab3d1ed8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:50.140246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac848c9-d87c-4734-9aab-8f83ab3d1ed8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:50.148064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebf29d7-2c27-4035-81a0-81f46e96a1fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:50.149205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf29d7-2c27-4035-81a0-81f46e96a1fb service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:50.149811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf29d7-2c27-4035-81a0-81f46e96a1fb service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:50.150345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebf29d7-2c27-4035-81a0-81f46e96a1fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:50.153276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac848c9-d87c-4734-9aab-8f83ab3d1ed8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:50.154801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac848c9-d87c-4734-9aab-8f83ab3d1ed8 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:50.179909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 845/1675] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:04:50.183942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa255391-44ce-4cf7-a778-e40650c49a24 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:50.184530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa255391-44ce-4cf7-a778-e40650c49a24 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:50.187924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa255391-44ce-4cf7-a778-e40650c49a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:50.339526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-aa255391-44ce-4cf7-a778-e40650c49a24 None None] MFA Rules not processed for user `a1313e0937b842059f1e01480c98351c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:50.341145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aa255391-44ce-4cf7-a778-e40650c49a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:50.372448 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf29d7-2c27-4035-81a0-81f46e96a1fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:50.375352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebf29d7-2c27-4035-81a0-81f46e96a1fb service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:50.396070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 831/1676] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:49 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 699 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:04:50.397694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfdc36-8496-4f87-8e5e-4f987d86bf24 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:50.398162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfdc36-8496-4f87-8e5e-4f987d86bf24 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:50.398584 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddfdc36-8496-4f87-8e5e-4f987d86bf24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:50.420478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cddfdc36-8496-4f87-8e5e-4f987d86bf24 None None] MFA Rules not processed for user `430d2efc691e4aae9f96409b081c7e96`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:50.422715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cddfdc36-8496-4f87-8e5e-4f987d86bf24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:50.497562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 832/1677] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:50.498833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfdc36-8496-4f87-8e5e-4f987d86bf24 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:50.529301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddfdc36-8496-4f87-8e5e-4f987d86bf24 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:50.543634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188781db-4d88-45bd-a210-3eca3bde98a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:50.544993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188781db-4d88-45bd-a210-3eca3bde98a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:50.544993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188781db-4d88-45bd-a210-3eca3bde98a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:50.546360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188781db-4d88-45bd-a210-3eca3bde98a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:50.559594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 846/1678] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:50.566222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c3988-f641-4bd7-adcf-b918cb86a9ff None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:50.566515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c3988-f641-4bd7-adcf-b918cb86a9ff None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:50.566802 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c3988-f641-4bd7-adcf-b918cb86a9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:50.618238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c05c3988-f641-4bd7-adcf-b918cb86a9ff None None] MFA Rules not processed for user `a1313e0937b842059f1e01480c98351c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:50.619487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c05c3988-f641-4bd7-adcf-b918cb86a9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:50.646820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 847/1679] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:50.718976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:50.729573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188781db-4d88-45bd-a210-3eca3bde98a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:50.731068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188781db-4d88-45bd-a210-3eca3bde98a2 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:50.763674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 833/1680] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:51.041094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c3988-f641-4bd7-adcf-b918cb86a9ff None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:51.044892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c05c3988-f641-4bd7-adcf-b918cb86a9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:51.150464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:51.247452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c3988-f641-4bd7-adcf-b918cb86a9ff None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:51.258382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:51.259419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:51.259815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:51.260202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] PATH_INFO: `/v3/users/a852e401620f4660ae98ecc7437b07c1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:51.267797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a852e401620f4660ae98ecc7437b07c1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:51.270149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:51.291820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00dc81-d2c5-45e4-b2fd-1b1d224a9108 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:51.292116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00dc81-d2c5-45e4-b2fd-1b1d224a9108 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:51.292395 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f00dc81-d2c5-45e4-b2fd-1b1d224a9108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:51.355905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9f00dc81-d2c5-45e4-b2fd-1b1d224a9108 None None] MFA Rules not processed for user `3b7a90ca5e374fc882b644e34e0a3015`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:51.359915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9f00dc81-d2c5-45e4-b2fd-1b1d224a9108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:51.449681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a852e401620f4660ae98ecc7437b07c1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:51.451133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a852e401620f4660ae98ecc7437b07c1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:51.480974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a852e401620f4660ae98ecc7437b07c1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:51.513729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a852e401620f4660ae98ecc7437b07c1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:51.516877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a852e401620f4660ae98ecc7437b07c1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:51.535301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 834/1681] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:04:51.599906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 848/1682] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:04:51 2022] DELETE /identity/v3/users/a852e401620f4660ae98ecc7437b07c1 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:51.601450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2e89-04e0-4652-89e4-126443247ccf None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:51.601753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2e89-04e0-4652-89e4-126443247ccf None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:51.602034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe2e89-04e0-4652-89e4-126443247ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:51.654036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-08fe2e89-04e0-4652-89e4-126443247ccf None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:51.655450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-08fe2e89-04e0-4652-89e4-126443247ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:51.810978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 835/1683] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:51.824089 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:51.825841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:51.949166 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd554d64-6270-4422-8e1f-feccd6f70797 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:51.960017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4748c989-75bf-422f-accc-0062762efc7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:51.960937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748c989-75bf-422f-accc-0062762efc7d service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:51.961368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748c989-75bf-422f-accc-0062762efc7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:51.962464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4748c989-75bf-422f-accc-0062762efc7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:51.990195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465e695-1138-4fd3-9c5c-0f7dd04e6d9b None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:51.990563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465e695-1138-4fd3-9c5c-0f7dd04e6d9b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:51.990858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2465e695-1138-4fd3-9c5c-0f7dd04e6d9b None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:51.992709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 836/1684] 10.222.0.13 () {54 vars in 912 bytes} [Sun Jun 26 03:04:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 26 03:04:51.994051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748c989-75bf-422f-accc-0062762efc7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:51.995589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4748c989-75bf-422f-accc-0062762efc7d service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:51.999913 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cbaf8-1a24-4896-9b44-b11e246edc3c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:52.000206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cbaf8-1a24-4896-9b44-b11e246edc3c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:52.001050 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cbaf8-1a24-4896-9b44-b11e246edc3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:52.034743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 849/1685] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:52.097274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f581ce-5061-413a-bd19-50658ca98c37 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:52.097652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f581ce-5061-413a-bd19-50658ca98c37 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:52.097940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f581ce-5061-413a-bd19-50658ca98c37 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:52.100449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 850/1686] 10.222.0.13 () {54 vars in 912 bytes} [Sun Jun 26 03:04:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:04:52.111497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d25b96-4e4c-48cb-b3eb-be58535af324 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:52.111497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d25b96-4e4c-48cb-b3eb-be58535af324 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:52.111497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d25b96-4e4c-48cb-b3eb-be58535af324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:52.111768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e08cbaf8-1a24-4896-9b44-b11e246edc3c None None] MFA Rules not processed for user `8ff708d6758d4f7b913e763824f3b791`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:52.117957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e08cbaf8-1a24-4896-9b44-b11e246edc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:52.136770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 837/1687] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:04:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:04:52.148592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cbaf8-1a24-4896-9b44-b11e246edc3c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:52.153770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e08cbaf8-1a24-4896-9b44-b11e246edc3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:52.160314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e6d25b96-4e4c-48cb-b3eb-be58535af324 None None] MFA Rules not processed for user `8ff708d6758d4f7b913e763824f3b791`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:52.161480 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e6d25b96-4e4c-48cb-b3eb-be58535af324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:52.181236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cbaf8-1a24-4896-9b44-b11e246edc3c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:52.183245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 851/1688] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:04:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:52.188982 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09184dd-9ea1-437d-9969-cb57a4bf7e0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:52.192925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09184dd-9ea1-437d-9969-cb57a4bf7e0a service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:52.193318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09184dd-9ea1-437d-9969-cb57a4bf7e0a service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:52.193607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09184dd-9ea1-437d-9969-cb57a4bf7e0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:52.194451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d25b96-4e4c-48cb-b3eb-be58535af324 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:52.204576 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d25b96-4e4c-48cb-b3eb-be58535af324 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:52.214500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09184dd-9ea1-437d-9969-cb57a4bf7e0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:52.215120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407e8a0-b907-48c6-8f27-8d3ba073b96e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:52.215783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407e8a0-b907-48c6-8f27-8d3ba073b96e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:52.216065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407e8a0-b907-48c6-8f27-8d3ba073b96e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:52.216339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407e8a0-b907-48c6-8f27-8d3ba073b96e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:52.219765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09184dd-9ea1-437d-9969-cb57a4bf7e0a service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:52.239323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407e8a0-b907-48c6-8f27-8d3ba073b96e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:52.241157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2407e8a0-b907-48c6-8f27-8d3ba073b96e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:52.248779 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 838/1689] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:04:52.275533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 852/1690] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:04:53.194417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09184dd-9ea1-437d-9969-cb57a4bf7e0a service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:53.198355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a09184dd-9ea1-437d-9969-cb57a4bf7e0a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:53.227137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09184dd-9ea1-437d-9969-cb57a4bf7e0a service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:53.238591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:53.239803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:53.241646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:53.242387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] PATH_INFO: `/v3/projects/db758d203db946e2a9b3393e1500f970` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:53.247059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] RBAC: Authorizing `identity:delete_project(project_id=db758d203db946e2a9b3393e1500f970)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:53.249305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:53.362764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:53.370481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db758d203db946e2a9b3393e1500f970', 'request_id': 'req-1c435f12-8136-4ee4-8a14-7767c5f8c64b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:53.396871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db758d203db946e2a9b3393e1500f970', 'request_id': 'req-1c435f12-8136-4ee4-8a14-7767c5f8c64b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:53.398556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407e8a0-b907-48c6-8f27-8d3ba073b96e service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:53.400549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2407e8a0-b907-48c6-8f27-8d3ba073b96e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:53.438547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db758d203db946e2a9b3393e1500f970', 'request_id': 'req-1c435f12-8136-4ee4-8a14-7767c5f8c64b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:53.487486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] The token cache is being invalidate because project db758d203db946e2a9b3393e1500f970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:04:53.499162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:53.502082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1c435f12-8136-4ee4-8a14-7767c5f8c64b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:04:53.507868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 839/1691] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:04:53 2022] DELETE /identity/v3/projects/db758d203db946e2a9b3393e1500f970 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:53.512724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c8bac-9be6-47fd-8c82-f22858049703 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:53.513024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c8bac-9be6-47fd-8c82-f22858049703 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:53.513310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6c8bac-9be6-47fd-8c82-f22858049703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:53.586135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:53.608086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4d6c8bac-9be6-47fd-8c82-f22858049703 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:53.609823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4d6c8bac-9be6-47fd-8c82-f22858049703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:53.696971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2407e8a0-b907-48c6-8f27-8d3ba073b96e service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:53.787136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 840/1692] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:53.788709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1760995-f808-42f6-ba52-8bac9995651a None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:53.789001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1760995-f808-42f6-ba52-8bac9995651a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:53.789279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1760995-f808-42f6-ba52-8bac9995651a None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:53.790810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 841/1693] 10.222.0.13 () {54 vars in 960 bytes} [Sun Jun 26 03:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:04:53.795975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaf85e-f5ab-4f98-a0a4-ec9acd649544 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:53.796269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaf85e-f5ab-4f98-a0a4-ec9acd649544 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:53.796563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffaf85e-f5ab-4f98-a0a4-ec9acd649544 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:53.855699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dffaf85e-f5ab-4f98-a0a4-ec9acd649544 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:53.856949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dffaf85e-f5ab-4f98-a0a4-ec9acd649544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:53.880233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bd972-e2e9-432c-9059-a06887f02a33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:53.880905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bd972-e2e9-432c-9059-a06887f02a33 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:53.881190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bd972-e2e9-432c-9059-a06887f02a33 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:53.881460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5bd972-e2e9-432c-9059-a06887f02a33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:53.887626 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2c5bd972-e2e9-432c-9059-a06887f02a33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:53.894749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 842/1694] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:53.894749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4ab87-dfac-4d22-be8d-acbaf37afb1a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:53.894749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4ab87-dfac-4d22-be8d-acbaf37afb1a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:53.895095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee4ab87-dfac-4d22-be8d-acbaf37afb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:53.929460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bee4ab87-dfac-4d22-be8d-acbaf37afb1a None None] MFA Rules not processed for user `19ce6f259c22441486bfc04811a271b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:53.930703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bee4ab87-dfac-4d22-be8d-acbaf37afb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:54.006776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5bd972-e2e9-432c-9059-a06887f02a33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:54.009370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5bd972-e2e9-432c-9059-a06887f02a33 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:54.025611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 853/1695] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:04:54.027121 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bd972-e2e9-432c-9059-a06887f02a33 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:54.028549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2c5bd972-e2e9-432c-9059-a06887f02a33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:54.047682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5bd972-e2e9-432c-9059-a06887f02a33 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:54.050263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 843/1696] 10.222.0.13 () {58 vars in 1057 bytes} [Sun Jun 26 03:04:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:54.051355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4ab87-dfac-4d22-be8d-acbaf37afb1a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:54.055782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b3701d-2b63-4369-9844-0e0d66434034 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:54.056459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3701d-2b63-4369-9844-0e0d66434034 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:54.056750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3701d-2b63-4369-9844-0e0d66434034 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:54.057032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b3701d-2b63-4369-9844-0e0d66434034 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:54.059043 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d0b3701d-2b63-4369-9844-0e0d66434034 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:54.090637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee4ab87-dfac-4d22-be8d-acbaf37afb1a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:54.099130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383bd94-8a9b-4d87-935e-fd5cd3c1d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:54.100630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383bd94-8a9b-4d87-935e-fd5cd3c1d63a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:54.100630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383bd94-8a9b-4d87-935e-fd5cd3c1d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:54.101042 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3383bd94-8a9b-4d87-935e-fd5cd3c1d63a admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:54.101861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383bd94-8a9b-4d87-935e-fd5cd3c1d63a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:04:54.102378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383bd94-8a9b-4d87-935e-fd5cd3c1d63a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:54.104173 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3383bd94-8a9b-4d87-935e-fd5cd3c1d63a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:54.134374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 844/1697] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:04:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:04:54.134374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383bd94-8a9b-4d87-935e-fd5cd3c1d63a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:54.183804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3701d-2b63-4369-9844-0e0d66434034 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:54.185428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3383bd94-8a9b-4d87-935e-fd5cd3c1d63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:54.185640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b3701d-2b63-4369-9844-0e0d66434034 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:54.210437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 854/1698] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:54.223520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8aa6d7-f286-43d3-8771-789111ed4bcb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:54.223835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8aa6d7-f286-43d3-8771-789111ed4bcb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:54.224105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8aa6d7-f286-43d3-8771-789111ed4bcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:54.251508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3383bd94-8a9b-4d87-935e-fd5cd3c1d63a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:54.257470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9f8aa6d7-f286-43d3-8771-789111ed4bcb None None] MFA Rules not processed for user `95793ccb15844069890363d983f1680d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:54.262500 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9f8aa6d7-f286-43d3-8771-789111ed4bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:54.267281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:54.283074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec7879-b159-40b5-b238-211855df869d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:54.283968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec7879-b159-40b5-b238-211855df869d service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:54.284400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec7879-b159-40b5-b238-211855df869d service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:54.284827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ec7879-b159-40b5-b238-211855df869d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:54.291363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-95ec7879-b159-40b5-b238-211855df869d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:54.338111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec7879-b159-40b5-b238-211855df869d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:54.339761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ec7879-b159-40b5-b238-211855df869d service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:54.368096 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 845/1699] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:54.369226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec7879-b159-40b5-b238-211855df869d service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:54.408861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ec7879-b159-40b5-b238-211855df869d service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:54.423335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5aa0-ac09-42d1-9986-ed93a5c75918 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:54.425489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5aa0-ac09-42d1-9986-ed93a5c75918 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:54.425995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5aa0-ac09-42d1-9986-ed93a5c75918 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:54.428227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5aa0-ac09-42d1-9986-ed93a5c75918 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:54.444490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 855/1700] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:54.456244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d5aa0-ac09-42d1-9986-ed93a5c75918 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:54.459564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8aa6d7-f286-43d3-8771-789111ed4bcb None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:54.461742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d5aa0-ac09-42d1-9986-ed93a5c75918 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:54.491911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 846/1701] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:54.510530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8aa6d7-f286-43d3-8771-789111ed4bcb None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:54.527698 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6555d-e2f6-498f-9cbf-b4f41aece200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:54.528299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6555d-e2f6-498f-9cbf-b4f41aece200 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:54.528587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6555d-e2f6-498f-9cbf-b4f41aece200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:54.528870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b6555d-e2f6-498f-9cbf-b4f41aece200 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:54.529609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6555d-e2f6-498f-9cbf-b4f41aece200 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:54.532158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b6555d-e2f6-498f-9cbf-b4f41aece200 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:54.659289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5aa0-ac09-42d1-9986-ed93a5c75918 service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:54.660197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 856/1702] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:54 2022] POST /identity/v3/projects => generated 363 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:54.699984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5aa0-ac09-42d1-9986-ed93a5c75918 service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:54.798684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:55.037234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee393e7-e5d5-4623-b575-c47000559a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:55.037943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee393e7-e5d5-4623-b575-c47000559a0d admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:55.038286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee393e7-e5d5-4623-b575-c47000559a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:55.038567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee393e7-e5d5-4623-b575-c47000559a0d admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:55.073312 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee393e7-e5d5-4623-b575-c47000559a0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:55.073312 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee393e7-e5d5-4623-b575-c47000559a0d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:55.141722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6555d-e2f6-498f-9cbf-b4f41aece200 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:55.149529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 847/1703] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:54 2022] POST /identity/v3/users => generated 282 bytes in 491 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:04:55.157408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b6555d-e2f6-498f-9cbf-b4f41aece200 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:55.196876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8932f-98ae-41f4-8524-32ebdc5a6b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:55.197574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8932f-98ae-41f4-8524-32ebdc5a6b41 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:55.197860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8932f-98ae-41f4-8524-32ebdc5a6b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:55.198167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc8932f-98ae-41f4-8524-32ebdc5a6b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:55.198912 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8932f-98ae-41f4-8524-32ebdc5a6b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:55.200542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc8932f-98ae-41f4-8524-32ebdc5a6b41 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:55.252423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee393e7-e5d5-4623-b575-c47000559a0d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:55.254176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 857/1704] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:04:55.272011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee393e7-e5d5-4623-b575-c47000559a0d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:55.285659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4346c4f-c98f-4833-ba21-caae60b898c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:55.286593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4346c4f-c98f-4833-ba21-caae60b898c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:55.286888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4346c4f-c98f-4833-ba21-caae60b898c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:55.287401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4346c4f-c98f-4833-ba21-caae60b898c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:55.288717 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4346c4f-c98f-4833-ba21-caae60b898c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:55.294807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4346c4f-c98f-4833-ba21-caae60b898c9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:55.309880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 848/1705] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:04:55.314293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8932f-98ae-41f4-8524-32ebdc5a6b41 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:55.346551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc8932f-98ae-41f4-8524-32ebdc5a6b41 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:55.382839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898c438-e51a-41de-8912-e280f22818d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:55.386958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898c438-e51a-41de-8912-e280f22818d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:55.388224 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898c438-e51a-41de-8912-e280f22818d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:55.388790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898c438-e51a-41de-8912-e280f22818d2 admin admin] PATH_INFO: `/v3/projects/76d729228a9b462d8122cab38c5fffb8/users/510ec43674cc414d9d52b48b18f434d9/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:55.432288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898c438-e51a-41de-8912-e280f22818d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76d729228a9b462d8122cab38c5fffb8, user_id=510ec43674cc414d9d52b48b18f434d9, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:55.437210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898c438-e51a-41de-8912-e280f22818d2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:55.482400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 858/1706] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:55 2022] PUT /identity/v3/projects/76d729228a9b462d8122cab38c5fffb8/users/510ec43674cc414d9d52b48b18f434d9/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:55.492224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a9c3f-432c-4e2e-9e48-adb3c7c6dce5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:55.493289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a9c3f-432c-4e2e-9e48-adb3c7c6dce5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:55.493585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89a9c3f-432c-4e2e-9e48-adb3c7c6dce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:55.573211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d89a9c3f-432c-4e2e-9e48-adb3c7c6dce5 None None] MFA Rules not processed for user `510ec43674cc414d9d52b48b18f434d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:55.575643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d89a9c3f-432c-4e2e-9e48-adb3c7c6dce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:55.895354 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 849/1707] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:55 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:55.896610 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898c438-e51a-41de-8912-e280f22818d2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:56.038679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898c438-e51a-41de-8912-e280f22818d2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:56.049054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944a6e-d122-4d1b-8911-c71231a25400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:56.054518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944a6e-d122-4d1b-8911-c71231a25400 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:56.054810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944a6e-d122-4d1b-8911-c71231a25400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:56.055117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15944a6e-d122-4d1b-8911-c71231a25400 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:56.055820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944a6e-d122-4d1b-8911-c71231a25400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:56.062570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15944a6e-d122-4d1b-8911-c71231a25400 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:56.172979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a9c3f-432c-4e2e-9e48-adb3c7c6dce5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:56.177934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 859/1708] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:04:55 2022] POST /identity/v3/projects => generated 363 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:56.190950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89a9c3f-432c-4e2e-9e48-adb3c7c6dce5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:56.316496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5df9-51a2-42d5-b4c5-6bb508e66a60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:56.317245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5df9-51a2-42d5-b4c5-6bb508e66a60 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:56.317672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5df9-51a2-42d5-b4c5-6bb508e66a60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:56.318169 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b5df9-51a2-42d5-b4c5-6bb508e66a60 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:56.318885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b5df9-51a2-42d5-b4c5-6bb508e66a60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:56.321009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b5df9-51a2-42d5-b4c5-6bb508e66a60 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:56.351191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 850/1709] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:04:56 2022] POST /identity/v3/users => generated 288 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:04:56.354997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944a6e-d122-4d1b-8911-c71231a25400 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:56.371603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15944a6e-d122-4d1b-8911-c71231a25400 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:56.378336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53a198-185d-4b0a-a864-8f39eab38d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:56.382985 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53a198-185d-4b0a-a864-8f39eab38d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:56.383412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53a198-185d-4b0a-a864-8f39eab38d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:56.383829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f53a198-185d-4b0a-a864-8f39eab38d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:56.384982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53a198-185d-4b0a-a864-8f39eab38d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:56.387063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f53a198-185d-4b0a-a864-8f39eab38d7f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:56.405280 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 860/1710] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:04:56.409278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5df9-51a2-42d5-b4c5-6bb508e66a60 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:56.430774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b5df9-51a2-42d5-b4c5-6bb508e66a60 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:56.451589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c494f39-5eed-46b9-9c9f-101e70f75025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:56.455037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c494f39-5eed-46b9-9c9f-101e70f75025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:56.456197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c494f39-5eed-46b9-9c9f-101e70f75025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:56.457971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c494f39-5eed-46b9-9c9f-101e70f75025 admin admin] PATH_INFO: `/v3/projects/500c65f11b574677865da089f2934564/users/0d49d6137a1940139998357ddbc2a88e/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:56.499769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c494f39-5eed-46b9-9c9f-101e70f75025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=500c65f11b574677865da089f2934564, user_id=0d49d6137a1940139998357ddbc2a88e, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:56.503514 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c494f39-5eed-46b9-9c9f-101e70f75025 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:56.517334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:56.540475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 851/1711] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:56 2022] PUT /identity/v3/projects/500c65f11b574677865da089f2934564/users/0d49d6137a1940139998357ddbc2a88e/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:56.541150 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53a198-185d-4b0a-a864-8f39eab38d7f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:56.561995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f53a198-185d-4b0a-a864-8f39eab38d7f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:56.693640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:56.796818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff23d60e-5e59-4285-8e38-471f1afd7b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:56.797532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d60e-5e59-4285-8e38-471f1afd7b84 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:56.798433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d60e-5e59-4285-8e38-471f1afd7b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:56.798433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff23d60e-5e59-4285-8e38-471f1afd7b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:56.798798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23d60e-5e59-4285-8e38-471f1afd7b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:56.800407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff23d60e-5e59-4285-8e38-471f1afd7b84 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:56.860347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 861/1712] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:04:56 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:04:56.866147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c494f39-5eed-46b9-9c9f-101e70f75025 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:56.879281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c494f39-5eed-46b9-9c9f-101e70f75025 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:56.900955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e19b6-b03e-46c0-898a-fbbe1dafcc75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:56.902599 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e19b6-b03e-46c0-898a-fbbe1dafcc75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:56.905896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e19b6-b03e-46c0-898a-fbbe1dafcc75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:56.906299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e19b6-b03e-46c0-898a-fbbe1dafcc75 admin admin] PATH_INFO: `/v3/projects/500c65f11b574677865da089f2934564/users/0d49d6137a1940139998357ddbc2a88e/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:56.909397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e19b6-b03e-46c0-898a-fbbe1dafcc75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=500c65f11b574677865da089f2934564, user_id=0d49d6137a1940139998357ddbc2a88e, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:56.929976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e19b6-b03e-46c0-898a-fbbe1dafcc75 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:56.958483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 852/1713] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:04:56 2022] PUT /identity/v3/projects/500c65f11b574677865da089f2934564/users/0d49d6137a1940139998357ddbc2a88e/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:04:56.972422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b021e-50c5-4ccc-9ecd-c46f477d9efa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:56.972719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b021e-50c5-4ccc-9ecd-c46f477d9efa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:56.972998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06b021e-50c5-4ccc-9ecd-c46f477d9efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:57.106163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f06b021e-50c5-4ccc-9ecd-c46f477d9efa None None] MFA Rules not processed for user `0d49d6137a1940139998357ddbc2a88e`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:57.113605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f06b021e-50c5-4ccc-9ecd-c46f477d9efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:57.328953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517b194-ee68-4b94-92fb-4a3477033b54 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:57.329835 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517b194-ee68-4b94-92fb-4a3477033b54 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:57.330752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5517b194-ee68-4b94-92fb-4a3477033b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:57.339630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 862/1714] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:56 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:04:57.354957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b021e-50c5-4ccc-9ecd-c46f477d9efa None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:57.361944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f06b021e-50c5-4ccc-9ecd-c46f477d9efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:57.376298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5517b194-ee68-4b94-92fb-4a3477033b54 None None] MFA Rules not processed for user `a1313e0937b842059f1e01480c98351c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:57.379337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5517b194-ee68-4b94-92fb-4a3477033b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:57.494792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 853/1715] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:04:57.532716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06b021e-50c5-4ccc-9ecd-c46f477d9efa None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:57.554826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6cf1b-25a7-47c1-8d41-358c2bfa44bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:57.555516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cf1b-25a7-47c1-8d41-358c2bfa44bf service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:57.555803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cf1b-25a7-47c1-8d41-358c2bfa44bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:57.556083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6cf1b-25a7-47c1-8d41-358c2bfa44bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:57.606323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6cf1b-25a7-47c1-8d41-358c2bfa44bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:57.607805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6cf1b-25a7-47c1-8d41-358c2bfa44bf service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:57.667936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 863/1716] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:57 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:57.829643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517b194-ee68-4b94-92fb-4a3477033b54 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:04:57.851556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5517b194-ee68-4b94-92fb-4a3477033b54 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:04:57.858278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83429010-6d0a-4074-bb77-3cb7d03de0e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:04:57.859213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83429010-6d0a-4074-bb77-3cb7d03de0e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:57.859697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83429010-6d0a-4074-bb77-3cb7d03de0e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:57.860063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83429010-6d0a-4074-bb77-3cb7d03de0e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:58.063488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83429010-6d0a-4074-bb77-3cb7d03de0e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:04:58.068586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83429010-6d0a-4074-bb77-3cb7d03de0e3 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:04:58.114681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 854/1717] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:04:57 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:04:58.256595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fdbaf-87a9-4ed7-b54e-fd314768413a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:04:58.256906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fdbaf-87a9-4ed7-b54e-fd314768413a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:04:58.257208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3fdbaf-87a9-4ed7-b54e-fd314768413a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:04:58.299785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6f3fdbaf-87a9-4ed7-b54e-fd314768413a None None] MFA Rules not processed for user `3b7a90ca5e374fc882b644e34e0a3015`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:04:58.321489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6f3fdbaf-87a9-4ed7-b54e-fd314768413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:04:58.464995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:04:58.481928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 864/1718] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:04:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:01.186920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213ec35-cf2a-43aa-9706-7bbb67085f52 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:01.188409 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213ec35-cf2a-43aa-9706-7bbb67085f52 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:01.188974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213ec35-cf2a-43aa-9706-7bbb67085f52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:01.265474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b213ec35-cf2a-43aa-9706-7bbb67085f52 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:01.274310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b213ec35-cf2a-43aa-9706-7bbb67085f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:01.430822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 855/1719] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:05:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:01.782208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fdbaf-87a9-4ed7-b54e-fd314768413a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:01.783841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6f3fdbaf-87a9-4ed7-b54e-fd314768413a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:01.945838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3fdbaf-87a9-4ed7-b54e-fd314768413a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:01.961432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3d3c8-2a94-4172-9917-61149372a81b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:01.962737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3d3c8-2a94-4172-9917-61149372a81b service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:01.963464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3d3c8-2a94-4172-9917-61149372a81b service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:01.965582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3d3c8-2a94-4172-9917-61149372a81b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:01.987025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3d3c8-2a94-4172-9917-61149372a81b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:01.990595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3d3c8-2a94-4172-9917-61149372a81b service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:02.017535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 865/1720] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:05:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:05:03.930686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213ec35-cf2a-43aa-9706-7bbb67085f52 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:03.933598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b213ec35-cf2a-43aa-9706-7bbb67085f52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:04.003794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:04.096261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b213ec35-cf2a-43aa-9706-7bbb67085f52 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:04.105247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:04.106819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:04.107078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:04.107359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] PATH_INFO: `/v3/users/4137ef42899c4addb833f11d453c2243` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:04.109547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4137ef42899c4addb833f11d453c2243)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:04.111228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: WARNING py.warnings [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:05:04.111228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: warnings.warn(msg) Jun 26 03:05:04.111228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: Jun 26 03:05:04.114509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:04.300177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4137ef42899c4addb833f11d453c2243'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:04.301967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4137ef42899c4addb833f11d453c2243'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:04.303535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4137ef42899c4addb833f11d453c2243'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:04.349573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4137ef42899c4addb833f11d453c2243'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:04.364778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4137ef42899c4addb833f11d453c2243'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:04.411720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 856/1721] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:03 2022] DELETE /identity/v3/users/4137ef42899c4addb833f11d453c2243 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:04.417765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61cd5-91ba-4fa1-a050-99f1c46d1c3f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:04.418344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61cd5-91ba-4fa1-a050-99f1c46d1c3f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:04.418586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c61cd5-91ba-4fa1-a050-99f1c46d1c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:04.463188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-12c61cd5-91ba-4fa1-a050-99f1c46d1c3f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:04.474480 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-12c61cd5-91ba-4fa1-a050-99f1c46d1c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:04.637559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 866/1722] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:04.654434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:04.655914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:04.762316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdc3f3-646e-4041-807c-57bc43ff0ccf admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:04.766909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d12162-2dc6-4cad-bdb3-d9f292865565 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:04.767986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d12162-2dc6-4cad-bdb3-d9f292865565 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:04.768473 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d12162-2dc6-4cad-bdb3-d9f292865565 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:04.768945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d12162-2dc6-4cad-bdb3-d9f292865565 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:04.781580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d12162-2dc6-4cad-bdb3-d9f292865565 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:04.784046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d12162-2dc6-4cad-bdb3-d9f292865565 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:04.795651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 857/1723] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:05.181033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c61cd5-91ba-4fa1-a050-99f1c46d1c3f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:05.182495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-12c61cd5-91ba-4fa1-a050-99f1c46d1c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:05.198779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c61cd5-91ba-4fa1-a050-99f1c46d1c3f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:05.209537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:05.211584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:05.212291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:05.213977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] PATH_INFO: `/v3/projects/b9afae2a48e34178bdb031839b1781bf` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:05.215421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9afae2a48e34178bdb031839b1781bf)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:05.216614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: WARNING py.warnings [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 26 03:05:05.216614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: warnings.warn(msg) Jun 26 03:05:05.216614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: Jun 26 03:05:05.218446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:05.313598 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9afae2a48e34178bdb031839b1781bf', 'request_id': 'req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:05.320285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9afae2a48e34178bdb031839b1781bf', 'request_id': 'req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:05.332526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9afae2a48e34178bdb031839b1781bf', 'request_id': 'req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:05.361730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] The token cache is being invalidate because project b9afae2a48e34178bdb031839b1781bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:05.362127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:05.363193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:05.365351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 867/1724] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:05 2022] DELETE /identity/v3/projects/b9afae2a48e34178bdb031839b1781bf => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:05.368758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d12162-2dc6-4cad-bdb3-d9f292865565 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:05.380496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d12162-2dc6-4cad-bdb3-d9f292865565 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:05.506280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:05.533747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:05.543238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:05.543545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:05.543851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] PATH_INFO: `/v3/users/d1c93b02e4574506ad560ba840d3575e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:05.545775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1c93b02e4574506ad560ba840d3575e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:05.547955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:05.609342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1c93b02e4574506ad560ba840d3575e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:05.610759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1c93b02e4574506ad560ba840d3575e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:05.612933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1c93b02e4574506ad560ba840d3575e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:05.631429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1c93b02e4574506ad560ba840d3575e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:05.646004 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7680736e-b20f-4556-b22d-8e17c1354112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1c93b02e4574506ad560ba840d3575e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:05.674155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 858/1725] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:05 2022] DELETE /identity/v3/users/d1c93b02e4574506ad560ba840d3575e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:06.114134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:06.116160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:06.131303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af03d6f6-8718-4e6c-a2eb-2513e10d77a4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:06.143331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:06.203746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:06.205620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:06.206087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:06.206516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] PATH_INFO: `/v3/projects/766319fd756043ec966c9d6f82565c41` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:06.208054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=766319fd756043ec966c9d6f82565c41)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:06.209986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:06.259112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '766319fd756043ec966c9d6f82565c41', 'request_id': 'req-ef542c0c-e080-4c51-8eb6-4d552777e697'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:06.265162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '766319fd756043ec966c9d6f82565c41', 'request_id': 'req-ef542c0c-e080-4c51-8eb6-4d552777e697'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:06.272446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '766319fd756043ec966c9d6f82565c41', 'request_id': 'req-ef542c0c-e080-4c51-8eb6-4d552777e697'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:06.284427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] The token cache is being invalidate because project 766319fd756043ec966c9d6f82565c41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:06.284726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:06.285393 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef542c0c-e080-4c51-8eb6-4d552777e697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:06.287115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 868/1726] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:06 2022] DELETE /identity/v3/projects/766319fd756043ec966c9d6f82565c41 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:06.292047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb55d4-0dad-4468-9744-c466422c70c2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:06.292289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb55d4-0dad-4468-9744-c466422c70c2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:06.292573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbb55d4-0dad-4468-9744-c466422c70c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:06.330093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5fbb55d4-0dad-4468-9744-c466422c70c2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:06.331355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5fbb55d4-0dad-4468-9744-c466422c70c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:06.403053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 859/1727] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:06.411090 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff4880-ec24-4c0c-b54e-10d32bd21225 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:06.411585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff4880-ec24-4c0c-b54e-10d32bd21225 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:06.412106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff4880-ec24-4c0c-b54e-10d32bd21225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:06.433457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-adff4880-ec24-4c0c-b54e-10d32bd21225 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:06.434758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-adff4880-ec24-4c0c-b54e-10d32bd21225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:06.444107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 869/1728] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:06.448067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb55d4-0dad-4468-9744-c466422c70c2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:06.458190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbb55d4-0dad-4468-9744-c466422c70c2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:06.463477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b0eb4-fa2e-47d8-bcd3-54bfa3a040f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:06.464340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b0eb4-fa2e-47d8-bcd3-54bfa3a040f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:06.464778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b0eb4-fa2e-47d8-bcd3-54bfa3a040f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:06.465215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8b0eb4-fa2e-47d8-bcd3-54bfa3a040f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:06.466127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b0eb4-fa2e-47d8-bcd3-54bfa3a040f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:06.466606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b0eb4-fa2e-47d8-bcd3-54bfa3a040f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:06.470202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8b0eb4-fa2e-47d8-bcd3-54bfa3a040f9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:06.484858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 860/1729] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:05:06.489189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff4880-ec24-4c0c-b54e-10d32bd21225 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:06.499177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff4880-ec24-4c0c-b54e-10d32bd21225 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:06.505201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805eaa0-c60e-4486-a988-bf580ec7ec5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:06.505983 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805eaa0-c60e-4486-a988-bf580ec7ec5e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:06.506414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805eaa0-c60e-4486-a988-bf580ec7ec5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:06.506791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805eaa0-c60e-4486-a988-bf580ec7ec5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:06.508006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805eaa0-c60e-4486-a988-bf580ec7ec5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:06.509855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805eaa0-c60e-4486-a988-bf580ec7ec5e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:06.542609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 870/1730] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:06 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:06.547351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b0eb4-fa2e-47d8-bcd3-54bfa3a040f9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:06.559431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8b0eb4-fa2e-47d8-bcd3-54bfa3a040f9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:06.672514 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcabd2a-fbff-43c2-bda8-01f725bbc556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:06.673536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcabd2a-fbff-43c2-bda8-01f725bbc556 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:06.673920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcabd2a-fbff-43c2-bda8-01f725bbc556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:06.674344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcabd2a-fbff-43c2-bda8-01f725bbc556 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:06.675334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcabd2a-fbff-43c2-bda8-01f725bbc556 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:06.681678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcabd2a-fbff-43c2-bda8-01f725bbc556 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:06.696074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:06.739702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 861/1731] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:06 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:06.740938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805eaa0-c60e-4486-a988-bf580ec7ec5e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:06.754414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805eaa0-c60e-4486-a988-bf580ec7ec5e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:06.763787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fc837-ce72-4b89-a1a7-d003633db17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:06.766515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fc837-ce72-4b89-a1a7-d003633db17e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:06.767942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fc837-ce72-4b89-a1a7-d003633db17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:06.770421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0fc837-ce72-4b89-a1a7-d003633db17e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:06.773668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fc837-ce72-4b89-a1a7-d003633db17e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:06.778781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0fc837-ce72-4b89-a1a7-d003633db17e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:06.831041 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 871/1732] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:06.834737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcabd2a-fbff-43c2-bda8-01f725bbc556 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:06.847007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcabd2a-fbff-43c2-bda8-01f725bbc556 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:06.858278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca61dfb-9750-446a-a2d0-e4d08eb67811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:06.859279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca61dfb-9750-446a-a2d0-e4d08eb67811 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:06.860455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca61dfb-9750-446a-a2d0-e4d08eb67811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:06.861087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca61dfb-9750-446a-a2d0-e4d08eb67811 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:06.862070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca61dfb-9750-446a-a2d0-e4d08eb67811 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:06.863895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca61dfb-9750-446a-a2d0-e4d08eb67811 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:06.891720 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fc837-ce72-4b89-a1a7-d003633db17e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:06.897699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 862/1733] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:06.907461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0fc837-ce72-4b89-a1a7-d003633db17e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:06.918194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed706-e111-4bc8-bced-36f484290d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:06.919026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed706-e111-4bc8-bced-36f484290d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:06.922479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed706-e111-4bc8-bced-36f484290d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:06.922891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed706-e111-4bc8-bced-36f484290d87 admin admin] PATH_INFO: `/v3/projects/935b7d68d52b4a87800f5159bdc4e896/users/e15c8326e8f54baeba1b831818e6bce8/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:06.963306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ed706-e111-4bc8-bced-36f484290d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=935b7d68d52b4a87800f5159bdc4e896, user_id=e15c8326e8f54baeba1b831818e6bce8, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:06.968115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22ed706-e111-4bc8-bced-36f484290d87 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:06.992665 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 872/1734] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:06 2022] PUT /identity/v3/projects/935b7d68d52b4a87800f5159bdc4e896/users/e15c8326e8f54baeba1b831818e6bce8/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:07.004915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d045c0-0bf3-44e5-9aa1-1ebf95dc1eae None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:07.005217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d045c0-0bf3-44e5-9aa1-1ebf95dc1eae None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:07.005510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d045c0-0bf3-44e5-9aa1-1ebf95dc1eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:07.048685 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-74d045c0-0bf3-44e5-9aa1-1ebf95dc1eae None None] MFA Rules not processed for user `e15c8326e8f54baeba1b831818e6bce8`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:07.050285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-74d045c0-0bf3-44e5-9aa1-1ebf95dc1eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:07.183971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 863/1735] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:07.189867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed706-e111-4bc8-bced-36f484290d87 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:07.260049 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:07.284973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed706-e111-4bc8-bced-36f484290d87 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:07.290065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad025c-1cdf-4bcd-8151-62ee84958d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:07.290795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad025c-1cdf-4bcd-8151-62ee84958d32 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:07.291083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad025c-1cdf-4bcd-8151-62ee84958d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:07.291360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dad025c-1cdf-4bcd-8151-62ee84958d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:07.292058 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad025c-1cdf-4bcd-8151-62ee84958d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:07.293984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dad025c-1cdf-4bcd-8151-62ee84958d32 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:07.370145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 873/1736] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:07 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:07.374072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d045c0-0bf3-44e5-9aa1-1ebf95dc1eae None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:07.391771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d045c0-0bf3-44e5-9aa1-1ebf95dc1eae None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:07.558950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8879d-7039-40a7-bea5-8685dca8b525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:07.559614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8879d-7039-40a7-bea5-8685dca8b525 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:07.559614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8879d-7039-40a7-bea5-8685dca8b525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:07.560057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8879d-7039-40a7-bea5-8685dca8b525 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:07.561701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8879d-7039-40a7-bea5-8685dca8b525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:07.567361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b8879d-7039-40a7-bea5-8685dca8b525 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:07.618823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 864/1737] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:07 2022] POST /identity/v3/users => generated 300 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:07.623938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad025c-1cdf-4bcd-8151-62ee84958d32 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:07.637827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dad025c-1cdf-4bcd-8151-62ee84958d32 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:07.645300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d6575-4224-40bb-955f-9215c57a4433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:07.646260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d6575-4224-40bb-955f-9215c57a4433 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:07.647019 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d6575-4224-40bb-955f-9215c57a4433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:07.647394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1d6575-4224-40bb-955f-9215c57a4433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:07.648330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d6575-4224-40bb-955f-9215c57a4433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:07.650841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1d6575-4224-40bb-955f-9215c57a4433 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:07.675416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 874/1738] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:07.679195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8879d-7039-40a7-bea5-8685dca8b525 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:07.702537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b8879d-7039-40a7-bea5-8685dca8b525 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:07.715704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1d5d2-dd0e-4452-ad43-56dbf4ac86c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:07.717197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1d5d2-dd0e-4452-ad43-56dbf4ac86c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:07.717197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1d5d2-dd0e-4452-ad43-56dbf4ac86c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:07.717197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da1d5d2-dd0e-4452-ad43-56dbf4ac86c5 admin admin] PATH_INFO: `/v3/projects/9bebf16bc97d46a081defcbd4585fbf6/users/04a594c59af8426da7dce0bde84c52ae/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:07.736325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1d5d2-dd0e-4452-ad43-56dbf4ac86c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bebf16bc97d46a081defcbd4585fbf6, user_id=04a594c59af8426da7dce0bde84c52ae, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:07.739198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da1d5d2-dd0e-4452-ad43-56dbf4ac86c5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:07.761157 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 865/1739] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:07 2022] PUT /identity/v3/projects/9bebf16bc97d46a081defcbd4585fbf6/users/04a594c59af8426da7dce0bde84c52ae/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:07.762095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d6575-4224-40bb-955f-9215c57a4433 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:07.778158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1d6575-4224-40bb-955f-9215c57a4433 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:07.925253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaab9eb-82f6-4a47-967d-ee6c2a6a2973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:07.933220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaab9eb-82f6-4a47-967d-ee6c2a6a2973 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:07.933511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaab9eb-82f6-4a47-967d-ee6c2a6a2973 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:07.934341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaab9eb-82f6-4a47-967d-ee6c2a6a2973 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:07.936873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaab9eb-82f6-4a47-967d-ee6c2a6a2973 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:07.943811 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaab9eb-82f6-4a47-967d-ee6c2a6a2973 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:07.999075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 875/1740] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:08.003649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1d5d2-dd0e-4452-ad43-56dbf4ac86c5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:08.015502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da1d5d2-dd0e-4452-ad43-56dbf4ac86c5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:08.023011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983dc116-38ef-4e82-bc64-45616d0d1890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:08.024153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dc116-38ef-4e82-bc64-45616d0d1890 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:08.024433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dc116-38ef-4e82-bc64-45616d0d1890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:08.024862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983dc116-38ef-4e82-bc64-45616d0d1890 admin admin] PATH_INFO: `/v3/projects/9bebf16bc97d46a081defcbd4585fbf6/users/04a594c59af8426da7dce0bde84c52ae/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:08.033871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983dc116-38ef-4e82-bc64-45616d0d1890 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bebf16bc97d46a081defcbd4585fbf6, user_id=04a594c59af8426da7dce0bde84c52ae, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:08.036739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983dc116-38ef-4e82-bc64-45616d0d1890 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:08.056981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 866/1741] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:08 2022] PUT /identity/v3/projects/9bebf16bc97d46a081defcbd4585fbf6/users/04a594c59af8426da7dce0bde84c52ae/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:08.067383 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11f63d-ef56-414b-8d95-f1c5054bbc8a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:08.067696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11f63d-ef56-414b-8d95-f1c5054bbc8a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:08.067982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e11f63d-ef56-414b-8d95-f1c5054bbc8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:08.146449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1e11f63d-ef56-414b-8d95-f1c5054bbc8a None None] MFA Rules not processed for user `04a594c59af8426da7dce0bde84c52ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:08.147792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1e11f63d-ef56-414b-8d95-f1c5054bbc8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:08.282993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 876/1742] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:08.293447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983dc116-38ef-4e82-bc64-45616d0d1890 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:08.296673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-983dc116-38ef-4e82-bc64-45616d0d1890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:08.334227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:08.481798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983dc116-38ef-4e82-bc64-45616d0d1890 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:08.493415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6043a-71b8-4557-8bf8-60a7d3e24019 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:08.496806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6043a-71b8-4557-8bf8-60a7d3e24019 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:08.497250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6043a-71b8-4557-8bf8-60a7d3e24019 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:08.497692 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6043a-71b8-4557-8bf8-60a7d3e24019 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:08.519624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6043a-71b8-4557-8bf8-60a7d3e24019 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:08.521526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a6043a-71b8-4557-8bf8-60a7d3e24019 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:08.549323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 867/1743] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:08.634911 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11f63d-ef56-414b-8d95-f1c5054bbc8a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:08.706547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:08.748193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e11f63d-ef56-414b-8d95-f1c5054bbc8a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:08.754221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:08.755096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:08.755377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:08.755652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] PATH_INFO: `/v3/users/e15c8326e8f54baeba1b831818e6bce8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:08.756851 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e15c8326e8f54baeba1b831818e6bce8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:08.759974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:08.818377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e15c8326e8f54baeba1b831818e6bce8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:08.819040 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e15c8326e8f54baeba1b831818e6bce8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:08.826728 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e15c8326e8f54baeba1b831818e6bce8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:08.835099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e15c8326e8f54baeba1b831818e6bce8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:08.835782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e15c8326e8f54baeba1b831818e6bce8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:08.855583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 877/1744] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:08 2022] DELETE /identity/v3/users/e15c8326e8f54baeba1b831818e6bce8 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:08.860623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5d753-8a17-4200-b0e7-735534703d23 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:08.861084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5d753-8a17-4200-b0e7-735534703d23 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:08.861190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c5d753-8a17-4200-b0e7-735534703d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:08.878691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d5c5d753-8a17-4200-b0e7-735534703d23 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:08.879889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d5c5d753-8a17-4200-b0e7-735534703d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:08.935826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 868/1745] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:08.945865 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:08.947315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:09.012965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418c174-16f9-4ea6-bf7d-c2cb41bb5b58 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:09.016825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ab191-1300-4f28-b794-a3bd895543ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:09.017739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ab191-1300-4f28-b794-a3bd895543ee service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:09.017861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ab191-1300-4f28-b794-a3bd895543ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:09.018272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1ab191-1300-4f28-b794-a3bd895543ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:09.027477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ab191-1300-4f28-b794-a3bd895543ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:09.029183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1ab191-1300-4f28-b794-a3bd895543ee service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:09.039419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 878/1746] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:09.427312 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5d753-8a17-4200-b0e7-735534703d23 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:09.428887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d5c5d753-8a17-4200-b0e7-735534703d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:09.440769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c5d753-8a17-4200-b0e7-735534703d23 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:09.445783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:09.446823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:09.447118 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:09.447401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] PATH_INFO: `/v3/projects/935b7d68d52b4a87800f5159bdc4e896` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:09.448850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=935b7d68d52b4a87800f5159bdc4e896)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:09.451119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:09.512104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '935b7d68d52b4a87800f5159bdc4e896', 'request_id': 'req-745e3e83-757b-41a5-845d-c52766cae9f4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:09.518955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '935b7d68d52b4a87800f5159bdc4e896', 'request_id': 'req-745e3e83-757b-41a5-845d-c52766cae9f4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:09.524771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '935b7d68d52b4a87800f5159bdc4e896', 'request_id': 'req-745e3e83-757b-41a5-845d-c52766cae9f4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:09.535739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] The token cache is being invalidate because project 935b7d68d52b4a87800f5159bdc4e896 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:09.536030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:09.536671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:09.538321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 869/1747] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:09 2022] DELETE /identity/v3/projects/935b7d68d52b4a87800f5159bdc4e896 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:09.542252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ab191-1300-4f28-b794-a3bd895543ee service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:09.552275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1ab191-1300-4f28-b794-a3bd895543ee service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:09.601852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:09.609227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:09.610319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:09.610803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:09.611322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] PATH_INFO: `/v3/users/04a594c59af8426da7dce0bde84c52ae` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:09.613044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a594c59af8426da7dce0bde84c52ae)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:09.615427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:09.664945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a594c59af8426da7dce0bde84c52ae'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:09.666471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a594c59af8426da7dce0bde84c52ae'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:09.675784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a594c59af8426da7dce0bde84c52ae'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:09.683177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a594c59af8426da7dce0bde84c52ae'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:09.683876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2d78eb03-7b6f-424d-9cd9-5bf61f824c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a594c59af8426da7dce0bde84c52ae'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:09.702196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 879/1748] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:09 2022] DELETE /identity/v3/users/04a594c59af8426da7dce0bde84c52ae => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:09.998718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:10.000163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:10.013441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745e3e83-757b-41a5-845d-c52766cae9f4 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:10.062956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:10.076113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:10.077035 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.077400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.077609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] PATH_INFO: `/v3/projects/9bebf16bc97d46a081defcbd4585fbf6` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.078864 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bebf16bc97d46a081defcbd4585fbf6)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:10.080793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:10.129878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bebf16bc97d46a081defcbd4585fbf6', 'request_id': 'req-d512bd44-4330-4f13-b3e8-3bd347914ac4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:10.136810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bebf16bc97d46a081defcbd4585fbf6', 'request_id': 'req-d512bd44-4330-4f13-b3e8-3bd347914ac4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:10.142477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bebf16bc97d46a081defcbd4585fbf6', 'request_id': 'req-d512bd44-4330-4f13-b3e8-3bd347914ac4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:10.153237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] The token cache is being invalidate because project 9bebf16bc97d46a081defcbd4585fbf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:10.153465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:10.154130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d512bd44-4330-4f13-b3e8-3bd347914ac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:10.156156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 870/1749] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:09 2022] DELETE /identity/v3/projects/9bebf16bc97d46a081defcbd4585fbf6 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:10.161835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e1e62-e2b9-45a5-905f-cfd119f89ce8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.162151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e1e62-e2b9-45a5-905f-cfd119f89ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.162430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e1e62-e2b9-45a5-905f-cfd119f89ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.180488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-154e1e62-e2b9-45a5-905f-cfd119f89ce8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:10.181669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-154e1e62-e2b9-45a5-905f-cfd119f89ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:10.238877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 880/1750] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:10.246525 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f7730-2796-4da9-9e3b-b56d99b2d8f8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.246815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f7730-2796-4da9-9e3b-b56d99b2d8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.247095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f7730-2796-4da9-9e3b-b56d99b2d8f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.265849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-df4f7730-2796-4da9-9e3b-b56d99b2d8f8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:10.267114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-df4f7730-2796-4da9-9e3b-b56d99b2d8f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:10.274884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 871/1751] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:10.279030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e1e62-e2b9-45a5-905f-cfd119f89ce8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:10.289108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e1e62-e2b9-45a5-905f-cfd119f89ce8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:10.293549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa510de-c520-4bcc-914c-59d7140a14f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:10.294234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa510de-c520-4bcc-914c-59d7140a14f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.294519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa510de-c520-4bcc-914c-59d7140a14f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.294797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa510de-c520-4bcc-914c-59d7140a14f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.295519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa510de-c520-4bcc-914c-59d7140a14f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:10.295857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa510de-c520-4bcc-914c-59d7140a14f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:10.297454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa510de-c520-4bcc-914c-59d7140a14f3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:10.309751 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 881/1752] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:05:10.313377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f7730-2796-4da9-9e3b-b56d99b2d8f8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:10.322832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f7730-2796-4da9-9e3b-b56d99b2d8f8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:10.328686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52953b-b19e-4aee-bb50-aa869380a015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:10.329347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52953b-b19e-4aee-bb50-aa869380a015 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.329628 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52953b-b19e-4aee-bb50-aa869380a015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.329904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f52953b-b19e-4aee-bb50-aa869380a015 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.330649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52953b-b19e-4aee-bb50-aa869380a015 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:10.332544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f52953b-b19e-4aee-bb50-aa869380a015 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:10.361490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 872/1753] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:10 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:10.365682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa510de-c520-4bcc-914c-59d7140a14f3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:10.375880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa510de-c520-4bcc-914c-59d7140a14f3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:10.433948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de384-38ca-4e7e-af2d-77f9be13db64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:10.434652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de384-38ca-4e7e-af2d-77f9be13db64 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.434938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de384-38ca-4e7e-af2d-77f9be13db64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.435216 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de384-38ca-4e7e-af2d-77f9be13db64 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.435904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901de384-38ca-4e7e-af2d-77f9be13db64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:10.437823 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901de384-38ca-4e7e-af2d-77f9be13db64 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:10.446684 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:10.460122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 882/1754] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:10 2022] POST /identity/v3/users => generated 283 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:10.464145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52953b-b19e-4aee-bb50-aa869380a015 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:10.475749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f52953b-b19e-4aee-bb50-aa869380a015 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:10.480526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870d295-53fc-4ffc-9727-2769dca6648a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:10.481225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870d295-53fc-4ffc-9727-2769dca6648a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.481509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870d295-53fc-4ffc-9727-2769dca6648a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.481785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870d295-53fc-4ffc-9727-2769dca6648a admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.482547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870d295-53fc-4ffc-9727-2769dca6648a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:10.484178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870d295-53fc-4ffc-9727-2769dca6648a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:10.497867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 873/1755] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:10.499143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de384-38ca-4e7e-af2d-77f9be13db64 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:10.509165 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de384-38ca-4e7e-af2d-77f9be13db64 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:10.514387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147c94e-4545-4787-9c69-0bdb9e1e44ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:10.515129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147c94e-4545-4787-9c69-0bdb9e1e44ac admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.515484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147c94e-4545-4787-9c69-0bdb9e1e44ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.515831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147c94e-4545-4787-9c69-0bdb9e1e44ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.516640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147c94e-4545-4787-9c69-0bdb9e1e44ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:10.518352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147c94e-4545-4787-9c69-0bdb9e1e44ac admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:10.529185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 883/1756] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:10.533938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870d295-53fc-4ffc-9727-2769dca6648a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:10.543931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870d295-53fc-4ffc-9727-2769dca6648a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:10.548784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556efcc-df2f-416d-8758-f93908c77b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:10.549670 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556efcc-df2f-416d-8758-f93908c77b95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.549956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556efcc-df2f-416d-8758-f93908c77b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.550271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0556efcc-df2f-416d-8758-f93908c77b95 admin admin] PATH_INFO: `/v3/projects/2c98a233e7ff485eaf7a7857180ebdf4/users/13b79531893a47e8b89c90d320370fe5/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.566610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556efcc-df2f-416d-8758-f93908c77b95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c98a233e7ff485eaf7a7857180ebdf4, user_id=13b79531893a47e8b89c90d320370fe5, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:10.569391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0556efcc-df2f-416d-8758-f93908c77b95 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:10.586128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 874/1757] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:10 2022] PUT /identity/v3/projects/2c98a233e7ff485eaf7a7857180ebdf4/users/13b79531893a47e8b89c90d320370fe5/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:10.595441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafeefc-810a-4f94-85e1-d9410461e6b3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.595920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafeefc-810a-4f94-85e1-d9410461e6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.596072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafeefc-810a-4f94-85e1-d9410461e6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.621812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6fafeefc-810a-4f94-85e1-d9410461e6b3 None None] MFA Rules not processed for user `13b79531893a47e8b89c90d320370fe5`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:10.623717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6fafeefc-810a-4f94-85e1-d9410461e6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:10.711078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 884/1758] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:10.715590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556efcc-df2f-416d-8758-f93908c77b95 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:10.784845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0556efcc-df2f-416d-8758-f93908c77b95 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:10.790852 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6be499-a1bc-4589-a34e-89a96c8b78fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:10.791950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be499-a1bc-4589-a34e-89a96c8b78fd admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:10.792432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be499-a1bc-4589-a34e-89a96c8b78fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:10.792892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6be499-a1bc-4589-a34e-89a96c8b78fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:10.793785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6be499-a1bc-4589-a34e-89a96c8b78fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:10.795969 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6be499-a1bc-4589-a34e-89a96c8b78fd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:10.818445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:10.838226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 875/1759] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:10 2022] POST /identity/v3/projects => generated 365 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:10.839042 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafeefc-810a-4f94-85e1-d9410461e6b3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:10.863612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafeefc-810a-4f94-85e1-d9410461e6b3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:11.090653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20897450-96c1-49c2-8413-f8f942f799ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:11.091655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20897450-96c1-49c2-8413-f8f942f799ca admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:11.091971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20897450-96c1-49c2-8413-f8f942f799ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:11.092274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20897450-96c1-49c2-8413-f8f942f799ca admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:11.092975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20897450-96c1-49c2-8413-f8f942f799ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:11.094979 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20897450-96c1-49c2-8413-f8f942f799ca admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:11.128752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 885/1760] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:10 2022] POST /identity/v3/users => generated 289 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:11.132818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6be499-a1bc-4589-a34e-89a96c8b78fd admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:11.155882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6be499-a1bc-4589-a34e-89a96c8b78fd admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:11.165057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afda4e4-2468-4eda-a97b-22aeba467db5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:11.165057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afda4e4-2468-4eda-a97b-22aeba467db5 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:11.165057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afda4e4-2468-4eda-a97b-22aeba467db5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:11.165057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afda4e4-2468-4eda-a97b-22aeba467db5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:11.165501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afda4e4-2468-4eda-a97b-22aeba467db5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:11.165501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afda4e4-2468-4eda-a97b-22aeba467db5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:11.182218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 876/1761] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:11.187345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20897450-96c1-49c2-8413-f8f942f799ca admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:11.203860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20897450-96c1-49c2-8413-f8f942f799ca admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:11.209666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a762793-e1dc-4866-b317-354478a1a316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:11.210924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a762793-e1dc-4866-b317-354478a1a316 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:11.211173 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a762793-e1dc-4866-b317-354478a1a316 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:11.211523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a762793-e1dc-4866-b317-354478a1a316 admin admin] PATH_INFO: `/v3/projects/21eb573811644b9b8dff23f3fe814bff/users/6e65d18e87b749e5bd9e6d67453aba4d/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:11.229838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a762793-e1dc-4866-b317-354478a1a316 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21eb573811644b9b8dff23f3fe814bff, user_id=6e65d18e87b749e5bd9e6d67453aba4d, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:11.233398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a762793-e1dc-4866-b317-354478a1a316 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:11.249673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 886/1762] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:11 2022] PUT /identity/v3/projects/21eb573811644b9b8dff23f3fe814bff/users/6e65d18e87b749e5bd9e6d67453aba4d/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:11.254299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afda4e4-2468-4eda-a97b-22aeba467db5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:11.265639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afda4e4-2468-4eda-a97b-22aeba467db5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:11.328038 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff9718-1494-44c5-ac68-c0f0cbf5cddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:11.328778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff9718-1494-44c5-ac68-c0f0cbf5cddd admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:11.329068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff9718-1494-44c5-ac68-c0f0cbf5cddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:11.329344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff9718-1494-44c5-ac68-c0f0cbf5cddd admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:11.330143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff9718-1494-44c5-ac68-c0f0cbf5cddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:11.331784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff9718-1494-44c5-ac68-c0f0cbf5cddd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:11.343158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 877/1763] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:11.347471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a762793-e1dc-4866-b317-354478a1a316 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:11.357596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a762793-e1dc-4866-b317-354478a1a316 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:11.362937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309b160-baa9-44ca-a8f9-d358f1ea46c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:11.363894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309b160-baa9-44ca-a8f9-d358f1ea46c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:11.364084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309b160-baa9-44ca-a8f9-d358f1ea46c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:11.364356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309b160-baa9-44ca-a8f9-d358f1ea46c1 admin admin] PATH_INFO: `/v3/projects/21eb573811644b9b8dff23f3fe814bff/users/6e65d18e87b749e5bd9e6d67453aba4d/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:11.366360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309b160-baa9-44ca-a8f9-d358f1ea46c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21eb573811644b9b8dff23f3fe814bff, user_id=6e65d18e87b749e5bd9e6d67453aba4d, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:11.368935 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309b160-baa9-44ca-a8f9-d358f1ea46c1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:11.380617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 887/1764] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:11 2022] PUT /identity/v3/projects/21eb573811644b9b8dff23f3fe814bff/users/6e65d18e87b749e5bd9e6d67453aba4d/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:11.391604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaebf7-9e5e-4ce6-950f-0f322241bf2e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:11.391604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaebf7-9e5e-4ce6-950f-0f322241bf2e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:11.391604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aaebf7-9e5e-4ce6-950f-0f322241bf2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:11.418030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-05aaebf7-9e5e-4ce6-950f-0f322241bf2e None None] MFA Rules not processed for user `6e65d18e87b749e5bd9e6d67453aba4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:11.419339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-05aaebf7-9e5e-4ce6-950f-0f322241bf2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:11.490249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 878/1765] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:11.501231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309b160-baa9-44ca-a8f9-d358f1ea46c1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:11.503181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e309b160-baa9-44ca-a8f9-d358f1ea46c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:11.533374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:11.565869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309b160-baa9-44ca-a8f9-d358f1ea46c1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:11.569450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc4b09-010e-4e1e-a233-90ac18d56ccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:11.570137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc4b09-010e-4e1e-a233-90ac18d56ccd service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:11.570421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc4b09-010e-4e1e-a233-90ac18d56ccd service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:11.570693 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc4b09-010e-4e1e-a233-90ac18d56ccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:11.579767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc4b09-010e-4e1e-a233-90ac18d56ccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:11.581172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc4b09-010e-4e1e-a233-90ac18d56ccd service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:11.592794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 888/1766] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:11.716475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aaebf7-9e5e-4ce6-950f-0f322241bf2e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:11.734013 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aaebf7-9e5e-4ce6-950f-0f322241bf2e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:11.738446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2785313-c201-4e2e-9eb9-24998ed4d8c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:11.739215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2785313-c201-4e2e-9eb9-24998ed4d8c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:11.739647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2785313-c201-4e2e-9eb9-24998ed4d8c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:11.739996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2785313-c201-4e2e-9eb9-24998ed4d8c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:11.806799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2785313-c201-4e2e-9eb9-24998ed4d8c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:11.808171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2785313-c201-4e2e-9eb9-24998ed4d8c4 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:11.817699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 879/1767] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:19.823088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326e26c-66cf-457d-9ea1-fd44c209caad None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:19.824303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326e26c-66cf-457d-9ea1-fd44c209caad None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:19.824782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4326e26c-66cf-457d-9ea1-fd44c209caad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:19.870751 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4326e26c-66cf-457d-9ea1-fd44c209caad None None] MFA Rules not processed for user `8ff708d6758d4f7b913e763824f3b791`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:19.872827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4326e26c-66cf-457d-9ea1-fd44c209caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:19.951697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 889/1768] 10.222.0.43 () {58 vars in 1015 bytes} [Sun Jun 26 03:05:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:20.055346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326e26c-66cf-457d-9ea1-fd44c209caad None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:20.056176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2785313-c201-4e2e-9eb9-24998ed4d8c4 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:20.057078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4326e26c-66cf-457d-9ea1-fd44c209caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:20.058894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c2785313-c201-4e2e-9eb9-24998ed4d8c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:20.069426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4326e26c-66cf-457d-9ea1-fd44c209caad None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:20.072150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2785313-c201-4e2e-9eb9-24998ed4d8c4 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:20.074186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346154a3-d870-4e76-98fa-f831e6c73d14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:20.074836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346154a3-d870-4e76-98fa-f831e6c73d14 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:20.075114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346154a3-d870-4e76-98fa-f831e6c73d14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:20.075390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346154a3-d870-4e76-98fa-f831e6c73d14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:20.077682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be3da9-f066-478d-897f-76d0d860ff40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:20.078370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be3da9-f066-478d-897f-76d0d860ff40 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:20.078747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be3da9-f066-478d-897f-76d0d860ff40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:20.079124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be3da9-f066-478d-897f-76d0d860ff40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:20.082849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346154a3-d870-4e76-98fa-f831e6c73d14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:20.084444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346154a3-d870-4e76-98fa-f831e6c73d14 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:20.085832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:20.088699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be3da9-f066-478d-897f-76d0d860ff40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:20.090266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be3da9-f066-478d-897f-76d0d860ff40 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:20.094075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 890/1769] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:20.096651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346154a3-d870-4e76-98fa-f831e6c73d14 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:20.097837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-346154a3-d870-4e76-98fa-f831e6c73d14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:20.100029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 880/1770] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:20.101353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be3da9-f066-478d-897f-76d0d860ff40 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:20.111451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346154a3-d870-4e76-98fa-f831e6c73d14 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:20.112046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be3da9-f066-478d-897f-76d0d860ff40 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:20.115284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973c5f64-85b4-477f-b1f3-b5dbae6df3b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:20.115949 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c5f64-85b4-477f-b1f3-b5dbae6df3b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:20.116236 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c5f64-85b4-477f-b1f3-b5dbae6df3b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:20.116505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973c5f64-85b4-477f-b1f3-b5dbae6df3b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:20.116878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910987ec-a3eb-4d31-94fd-6ca1f05449be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:20.117815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910987ec-a3eb-4d31-94fd-6ca1f05449be service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:20.118359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910987ec-a3eb-4d31-94fd-6ca1f05449be service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:20.118850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910987ec-a3eb-4d31-94fd-6ca1f05449be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:20.126651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c5f64-85b4-477f-b1f3-b5dbae6df3b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:20.128727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910987ec-a3eb-4d31-94fd-6ca1f05449be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:20.128860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973c5f64-85b4-477f-b1f3-b5dbae6df3b5 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:20.130673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910987ec-a3eb-4d31-94fd-6ca1f05449be service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:20.139882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 891/1771] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:20.140873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 881/1772] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:21.655322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc583a5-9bf8-4e99-a6ae-0e04392a35d1 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:21.656149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc583a5-9bf8-4e99-a6ae-0e04392a35d1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:21.657137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc583a5-9bf8-4e99-a6ae-0e04392a35d1 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:21.658622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 892/1773] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:05:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:05:21.667998 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cc4a7-a1b7-4eff-875b-8fe683e5e923 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:21.668524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cc4a7-a1b7-4eff-875b-8fe683e5e923 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:21.668524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084cc4a7-a1b7-4eff-875b-8fe683e5e923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:21.710214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-084cc4a7-a1b7-4eff-875b-8fe683e5e923 None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:21.712458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-084cc4a7-a1b7-4eff-875b-8fe683e5e923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:21.868624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 882/1774] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:05:21 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:23.054956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc583a5-9bf8-4e99-a6ae-0e04392a35d1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:23.064585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1bc583a5-9bf8-4e99-a6ae-0e04392a35d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:23.160069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc583a5-9bf8-4e99-a6ae-0e04392a35d1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:23.197920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43145fe5-04e5-4096-87f3-729fa11a3a78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:23.200553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43145fe5-04e5-4096-87f3-729fa11a3a78 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:23.200971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43145fe5-04e5-4096-87f3-729fa11a3a78 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:23.201339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43145fe5-04e5-4096-87f3-729fa11a3a78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:23.216032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43145fe5-04e5-4096-87f3-729fa11a3a78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:23.217866 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43145fe5-04e5-4096-87f3-729fa11a3a78 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:23.235363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 893/1775] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:24.332726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba958797-adf7-480d-88c3-fc63ee267b55 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:24.333228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba958797-adf7-480d-88c3-fc63ee267b55 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:24.333323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba958797-adf7-480d-88c3-fc63ee267b55 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:24.335242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 883/1776] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:05:24.364177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f971fc-3263-4c7c-a1e0-787b96da2605 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:24.364714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f971fc-3263-4c7c-a1e0-787b96da2605 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:24.364714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f971fc-3263-4c7c-a1e0-787b96da2605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:24.386234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d9f971fc-3263-4c7c-a1e0-787b96da2605 None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:24.387366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d9f971fc-3263-4c7c-a1e0-787b96da2605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:24.396543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 894/1777] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:05:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:24.593797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba958797-adf7-480d-88c3-fc63ee267b55 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:24.597116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ba958797-adf7-480d-88c3-fc63ee267b55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:24.649787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f971fc-3263-4c7c-a1e0-787b96da2605 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:24.666353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f971fc-3263-4c7c-a1e0-787b96da2605 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:24.679197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddf4e90-cdb3-4bcc-9278-5e624d5a785d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:24.680022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf4e90-cdb3-4bcc-9278-5e624d5a785d service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:24.680630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf4e90-cdb3-4bcc-9278-5e624d5a785d service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:24.681007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddf4e90-cdb3-4bcc-9278-5e624d5a785d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:24.695334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf4e90-cdb3-4bcc-9278-5e624d5a785d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:24.696848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddf4e90-cdb3-4bcc-9278-5e624d5a785d service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:24.714383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba958797-adf7-480d-88c3-fc63ee267b55 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:24.720136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:24.721164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:24.721419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:24.721701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] PATH_INFO: `/v3/users/510ec43674cc414d9d52b48b18f434d9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:24.723043 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=510ec43674cc414d9d52b48b18f434d9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:24.725791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:24.726183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 895/1778] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:24.801907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '510ec43674cc414d9d52b48b18f434d9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:24.802615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '510ec43674cc414d9d52b48b18f434d9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:24.803338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '510ec43674cc414d9d52b48b18f434d9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:24.818945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '510ec43674cc414d9d52b48b18f434d9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:24.829192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '510ec43674cc414d9d52b48b18f434d9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:24.839685 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:24.854177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 884/1779] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:24 2022] DELETE /identity/v3/users/510ec43674cc414d9d52b48b18f434d9 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:24.858846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0acb48-8eca-4bce-8dfc-4a1ab7680474 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:24.859238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0acb48-8eca-4bce-8dfc-4a1ab7680474 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:24.859690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0acb48-8eca-4bce-8dfc-4a1ab7680474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:24.886800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-eb0acb48-8eca-4bce-8dfc-4a1ab7680474 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:24.888044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eb0acb48-8eca-4bce-8dfc-4a1ab7680474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:24.927227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:24.930934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:25.039363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 896/1780] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:25.048347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0acb48-8eca-4bce-8dfc-4a1ab7680474 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:25.049844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eb0acb48-8eca-4bce-8dfc-4a1ab7680474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:25.081215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:25.155566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1966a4d-04cb-404d-878f-4cfa43056527 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:25.166975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:25.168615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:25.169134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:25.169650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] PATH_INFO: `/v3/users/13b79531893a47e8b89c90d320370fe5` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:25.171876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] RBAC: Authorizing `identity:delete_user(user_id=13b79531893a47e8b89c90d320370fe5)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:25.174961 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:25.190572 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0acb48-8eca-4bce-8dfc-4a1ab7680474 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:25.202828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8067126-f81c-421a-9529-e65aecd9c6b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:25.204051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067126-f81c-421a-9529-e65aecd9c6b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:25.204339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067126-f81c-421a-9529-e65aecd9c6b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:25.204624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8067126-f81c-421a-9529-e65aecd9c6b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:25.230012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8067126-f81c-421a-9529-e65aecd9c6b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:25.231513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8067126-f81c-421a-9529-e65aecd9c6b8 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:25.276087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 897/1781] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:25.286623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13b79531893a47e8b89c90d320370fe5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.289536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13b79531893a47e8b89c90d320370fe5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.290598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13b79531893a47e8b89c90d320370fe5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.309411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13b79531893a47e8b89c90d320370fe5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.322947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13b79531893a47e8b89c90d320370fe5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.354337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 885/1782] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:24 2022] DELETE /identity/v3/users/13b79531893a47e8b89c90d320370fe5 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:25.356388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb676aac-88d1-44c9-beaa-c766398a6470 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:25.356708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb676aac-88d1-44c9-beaa-c766398a6470 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:25.356999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb676aac-88d1-44c9-beaa-c766398a6470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:25.383500 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-eb676aac-88d1-44c9-beaa-c766398a6470 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:25.385652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eb676aac-88d1-44c9-beaa-c766398a6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:25.506324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 898/1783] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:25.522427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:25.524838 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:25.649154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1fe4d-9127-4e49-8ffd-e79f102f984b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:25.661845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24be9a3-57bc-4c69-83fc-ca0a1926b15c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:25.662588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24be9a3-57bc-4c69-83fc-ca0a1926b15c service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:25.662838 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24be9a3-57bc-4c69-83fc-ca0a1926b15c service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:25.663119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24be9a3-57bc-4c69-83fc-ca0a1926b15c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:25.682832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24be9a3-57bc-4c69-83fc-ca0a1926b15c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:25.684611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24be9a3-57bc-4c69-83fc-ca0a1926b15c service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:25.700113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb676aac-88d1-44c9-beaa-c766398a6470 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:25.701855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eb676aac-88d1-44c9-beaa-c766398a6470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:25.714190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 886/1784] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:25.721486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb676aac-88d1-44c9-beaa-c766398a6470 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:25.732746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:25.733820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:25.734138 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:25.735138 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] PATH_INFO: `/v3/projects/76d729228a9b462d8122cab38c5fffb8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:25.736593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76d729228a9b462d8122cab38c5fffb8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:25.759787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:25.825669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76d729228a9b462d8122cab38c5fffb8', 'request_id': 'req-c96adbeb-2316-4725-a397-920b989c3a52'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.836307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76d729228a9b462d8122cab38c5fffb8', 'request_id': 'req-c96adbeb-2316-4725-a397-920b989c3a52'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.849594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76d729228a9b462d8122cab38c5fffb8', 'request_id': 'req-c96adbeb-2316-4725-a397-920b989c3a52'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.864343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] The token cache is being invalidate because project 76d729228a9b462d8122cab38c5fffb8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:25.864790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.865543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.867463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 899/1785] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:25 2022] DELETE /identity/v3/projects/76d729228a9b462d8122cab38c5fffb8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:25.871508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24be9a3-57bc-4c69-83fc-ca0a1926b15c service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:25.881933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24be9a3-57bc-4c69-83fc-ca0a1926b15c service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:25.896648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:25.944351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:25.945229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:25.945512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:25.945787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] PATH_INFO: `/v3/users/0d49d6137a1940139998357ddbc2a88e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:25.947022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d49d6137a1940139998357ddbc2a88e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:25.948953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:25.996337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d49d6137a1940139998357ddbc2a88e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.996990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d49d6137a1940139998357ddbc2a88e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:25.997660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d49d6137a1940139998357ddbc2a88e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.010433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d49d6137a1940139998357ddbc2a88e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.019073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d49d6137a1940139998357ddbc2a88e'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.032617 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 887/1786] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:25 2022] DELETE /identity/v3/users/0d49d6137a1940139998357ddbc2a88e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:26.314542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:26.315970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:26.357231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96adbeb-2316-4725-a397-920b989c3a52 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:26.420860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:26.437177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:26.441146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:26.441406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:26.442072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:26.442354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:26.442608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] PATH_INFO: `/v3/projects/500c65f11b574677865da089f2934564` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:26.443996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=500c65f11b574677865da089f2934564)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:26.446028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:26.456618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c78b24-9b9f-40ac-96de-e788295ecf2e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:26.462074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:26.463118 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:26.463582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:26.463861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] PATH_INFO: `/v3/projects/2c98a233e7ff485eaf7a7857180ebdf4` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:26.465082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c98a233e7ff485eaf7a7857180ebdf4)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:26.467204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:26.524176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '500c65f11b574677865da089f2934564', 'request_id': 'req-3b04fafc-ee29-4817-b7a9-3fc031d605fe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.526607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c98a233e7ff485eaf7a7857180ebdf4', 'request_id': 'req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.530241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '500c65f11b574677865da089f2934564', 'request_id': 'req-3b04fafc-ee29-4817-b7a9-3fc031d605fe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.535581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c98a233e7ff485eaf7a7857180ebdf4', 'request_id': 'req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.537877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '500c65f11b574677865da089f2934564', 'request_id': 'req-3b04fafc-ee29-4817-b7a9-3fc031d605fe'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.541513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c98a233e7ff485eaf7a7857180ebdf4', 'request_id': 'req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.549110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] The token cache is being invalidate because project 500c65f11b574677865da089f2934564 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:26.549402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.550032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3b04fafc-ee29-4817-b7a9-3fc031d605fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.551927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 900/1787] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:26 2022] DELETE /identity/v3/projects/500c65f11b574677865da089f2934564 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:26.553021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] The token cache is being invalidate because project 2c98a233e7ff485eaf7a7857180ebdf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:26.553307 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.553954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.555522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 888/1788] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:26 2022] DELETE /identity/v3/projects/2c98a233e7ff485eaf7a7857180ebdf4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:26.556427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbbdf0-c8b0-4863-bef9-1563e08cf9a5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:26.556854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbbdf0-c8b0-4863-bef9-1563e08cf9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:26.557217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfbbdf0-c8b0-4863-bef9-1563e08cf9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:26.559033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:26.560334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:26.575518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ccfbbdf0-c8b0-4863-bef9-1563e08cf9a5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:26.576757 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ccfbbdf0-c8b0-4863-bef9-1563e08cf9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:26.580585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:26.631674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d55b7e-9a64-4dcd-9ec6-c24fc9f13bde admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:26.636950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:26.638008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:26.638619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:26.638767 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] PATH_INFO: `/v3/users/6e65d18e87b749e5bd9e6d67453aba4d` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:26.639859 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e65d18e87b749e5bd9e6d67453aba4d)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:26.641858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:26.650770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 901/1789] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:26.657970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bbee3-c93c-4c11-b491-f67854097436 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:26.658412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bbee3-c93c-4c11-b491-f67854097436 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:26.658699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059bbee3-c93c-4c11-b491-f67854097436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:26.680195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-059bbee3-c93c-4c11-b491-f67854097436 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:26.681408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-059bbee3-c93c-4c11-b491-f67854097436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:26.690615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 902/1790] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:26.695301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bbee3-c93c-4c11-b491-f67854097436 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:26.702145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e65d18e87b749e5bd9e6d67453aba4d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.703549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e65d18e87b749e5bd9e6d67453aba4d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.704384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e65d18e87b749e5bd9e6d67453aba4d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.705220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059bbee3-c93c-4c11-b491-f67854097436 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:26.710280 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51532e-bc6b-4acc-ba96-b751ae577f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:26.710954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51532e-bc6b-4acc-ba96-b751ae577f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:26.711241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51532e-bc6b-4acc-ba96-b751ae577f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:26.711522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d51532e-bc6b-4acc-ba96-b751ae577f87 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:26.712251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51532e-bc6b-4acc-ba96-b751ae577f87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:26.712676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51532e-bc6b-4acc-ba96-b751ae577f87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:26.714407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d51532e-bc6b-4acc-ba96-b751ae577f87 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:26.718501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e65d18e87b749e5bd9e6d67453aba4d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.728067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e65d18e87b749e5bd9e6d67453aba4d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:26.732543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 903/1791] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:05:26.733425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51532e-bc6b-4acc-ba96-b751ae577f87 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:26.735521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9d51532e-bc6b-4acc-ba96-b751ae577f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:26.749305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 889/1792] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:26 2022] DELETE /identity/v3/users/6e65d18e87b749e5bd9e6d67453aba4d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:26.749629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d51532e-bc6b-4acc-ba96-b751ae577f87 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:26.823393 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:26.835408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def06018-d903-44a2-9747-d4ce0de78e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:26.836482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def06018-d903-44a2-9747-d4ce0de78e84 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:26.836858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def06018-d903-44a2-9747-d4ce0de78e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:26.837288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def06018-d903-44a2-9747-d4ce0de78e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:26.838428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def06018-d903-44a2-9747-d4ce0de78e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:26.841322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def06018-d903-44a2-9747-d4ce0de78e84 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:26.882634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 904/1793] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:26 2022] POST /identity/v3/projects => generated 381 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:26.883262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:26.895165 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8a20f-50d3-4c7e-a291-7ab6d199c799 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:26.966759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82176ce9-f0d8-44ef-925c-f959bb7eeca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:26.967452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82176ce9-f0d8-44ef-925c-f959bb7eeca4 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:26.967744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82176ce9-f0d8-44ef-925c-f959bb7eeca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:26.968027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82176ce9-f0d8-44ef-925c-f959bb7eeca4 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:26.969361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82176ce9-f0d8-44ef-925c-f959bb7eeca4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:26.971757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82176ce9-f0d8-44ef-925c-f959bb7eeca4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:26.998138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 890/1794] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:26 2022] POST /identity/v3/users => generated 291 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:27.002270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def06018-d903-44a2-9747-d4ce0de78e84 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.015616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def06018-d903-44a2-9747-d4ce0de78e84 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.021982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5ee2-bd63-4246-b0d5-6a094a93e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.022416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5ee2-bd63-4246-b0d5-6a094a93e9a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.022705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5ee2-bd63-4246-b0d5-6a094a93e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.023057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c5ee2-bd63-4246-b0d5-6a094a93e9a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.024002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5ee2-bd63-4246-b0d5-6a094a93e9a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.025734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97c5ee2-bd63-4246-b0d5-6a094a93e9a2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.037854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 905/1795] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:27.041971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82176ce9-f0d8-44ef-925c-f959bb7eeca4 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.060721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82176ce9-f0d8-44ef-925c-f959bb7eeca4 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.066487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71a610-af91-44ad-b3ba-f340f95b1332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.067209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71a610-af91-44ad-b3ba-f340f95b1332 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.067446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71a610-af91-44ad-b3ba-f340f95b1332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.067719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce71a610-af91-44ad-b3ba-f340f95b1332 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.068492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71a610-af91-44ad-b3ba-f340f95b1332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.070174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce71a610-af91-44ad-b3ba-f340f95b1332 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.079383 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5ee2-bd63-4246-b0d5-6a094a93e9a2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.083628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f97c5ee2-bd63-4246-b0d5-6a094a93e9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:27.086227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 891/1796] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:27.090897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71a610-af91-44ad-b3ba-f340f95b1332 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.099013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c5ee2-bd63-4246-b0d5-6a094a93e9a2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.101352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce71a610-af91-44ad-b3ba-f340f95b1332 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.104201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.105203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.105586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.105984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] PATH_INFO: `/v3/projects/21eb573811644b9b8dff23f3fe814bff` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.107653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=21eb573811644b9b8dff23f3fe814bff)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.110002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.124348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e551fe-d10c-47c7-a81e-fe7b38bb464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.125906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e551fe-d10c-47c7-a81e-fe7b38bb464a admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.126244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e551fe-d10c-47c7-a81e-fe7b38bb464a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.126529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e551fe-d10c-47c7-a81e-fe7b38bb464a admin admin] PATH_INFO: `/v3/projects/1c66b7f14f9c44019c579e57913cc988/users/9e9111d750a24e77b461488fef77e3ee/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.141933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:27.153003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e551fe-d10c-47c7-a81e-fe7b38bb464a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c66b7f14f9c44019c579e57913cc988, user_id=9e9111d750a24e77b461488fef77e3ee, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.156211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e551fe-d10c-47c7-a81e-fe7b38bb464a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.174877 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 892/1797] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:27 2022] PUT /identity/v3/projects/1c66b7f14f9c44019c579e57913cc988/users/9e9111d750a24e77b461488fef77e3ee/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:27.185192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96bb34-1d32-43cd-bad2-677eafec464b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.185655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96bb34-1d32-43cd-bad2-677eafec464b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.186006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96bb34-1d32-43cd-bad2-677eafec464b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.191423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21eb573811644b9b8dff23f3fe814bff', 'request_id': 'req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:27.198663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21eb573811644b9b8dff23f3fe814bff', 'request_id': 'req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:27.208391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21eb573811644b9b8dff23f3fe814bff', 'request_id': 'req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:27.221908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-da96bb34-1d32-43cd-bad2-677eafec464b None None] MFA Rules not processed for user `9e9111d750a24e77b461488fef77e3ee`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:27.222439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] The token cache is being invalidate because project 21eb573811644b9b8dff23f3fe814bff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:27.222735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:27.223274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-da96bb34-1d32-43cd-bad2-677eafec464b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:27.223481 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a57d490-3aff-4a1f-acd2-ecd439cec7bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:27.225338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 906/1798] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:27 2022] DELETE /identity/v3/projects/21eb573811644b9b8dff23f3fe814bff => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:27.232152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff3884-2f92-443a-b657-6865b737ee4d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.232584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff3884-2f92-443a-b657-6865b737ee4d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.232868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bff3884-2f92-443a-b657-6865b737ee4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.259533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9bff3884-2f92-443a-b657-6865b737ee4d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:27.260977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9bff3884-2f92-443a-b657-6865b737ee4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:27.310027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 893/1799] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:27.311181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96bb34-1d32-43cd-bad2-677eafec464b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.313462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-da96bb34-1d32-43cd-bad2-677eafec464b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:27.329731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 907/1800] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:27.337570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79415e97-a887-42e1-add0-8dc32414b4a0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.337867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79415e97-a887-42e1-add0-8dc32414b4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.338190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79415e97-a887-42e1-add0-8dc32414b4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.356422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-79415e97-a887-42e1-add0-8dc32414b4a0 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:27.357934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-79415e97-a887-42e1-add0-8dc32414b4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:27.380659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 908/1801] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:27.384800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79415e97-a887-42e1-add0-8dc32414b4a0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.394272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79415e97-a887-42e1-add0-8dc32414b4a0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.400339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556f276-a692-43f3-95a8-1cd68307e670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.401073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556f276-a692-43f3-95a8-1cd68307e670 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.401436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556f276-a692-43f3-95a8-1cd68307e670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.401869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556f276-a692-43f3-95a8-1cd68307e670 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.402706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556f276-a692-43f3-95a8-1cd68307e670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:27.403109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556f276-a692-43f3-95a8-1cd68307e670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.404773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556f276-a692-43f3-95a8-1cd68307e670 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.412137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96bb34-1d32-43cd-bad2-677eafec464b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.416989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7cc3a-5209-40c6-b9ab-9efc38fdfb6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.417839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7cc3a-5209-40c6-b9ab-9efc38fdfb6c admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.417948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 909/1802] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:05:27.418134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7cc3a-5209-40c6-b9ab-9efc38fdfb6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.418498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa7cc3a-5209-40c6-b9ab-9efc38fdfb6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.419202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7cc3a-5209-40c6-b9ab-9efc38fdfb6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.421499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa7cc3a-5209-40c6-b9ab-9efc38fdfb6c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.422397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556f276-a692-43f3-95a8-1cd68307e670 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.432642 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556f276-a692-43f3-95a8-1cd68307e670 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.437448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cc335-a4b1-48e3-a07d-7025a6005140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.438686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cc335-a4b1-48e3-a07d-7025a6005140 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.439046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cc335-a4b1-48e3-a07d-7025a6005140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.439331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869cc335-a4b1-48e3-a07d-7025a6005140 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.440161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cc335-a4b1-48e3-a07d-7025a6005140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.442316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869cc335-a4b1-48e3-a07d-7025a6005140 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.453656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 894/1803] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/projects => generated 381 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:27.457613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7cc3a-5209-40c6-b9ab-9efc38fdfb6c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.479826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 910/1804] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:27.484050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cc335-a4b1-48e3-a07d-7025a6005140 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.498435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:27.523718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa7cc3a-5209-40c6-b9ab-9efc38fdfb6c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.533320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:27.575862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869cc335-a4b1-48e3-a07d-7025a6005140 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.581788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4da8aa-88c1-4af1-bfc1-c134f5548c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.582705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4da8aa-88c1-4af1-bfc1-c134f5548c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.582995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4da8aa-88c1-4af1-bfc1-c134f5548c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.583272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4da8aa-88c1-4af1-bfc1-c134f5548c19 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.583987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4da8aa-88c1-4af1-bfc1-c134f5548c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.585920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4da8aa-88c1-4af1-bfc1-c134f5548c19 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.590770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b1e37-0138-4c65-8ded-25a21468ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.591520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1e37-0138-4c65-8ded-25a21468ca71 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.591814 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1e37-0138-4c65-8ded-25a21468ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.592088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1e37-0138-4c65-8ded-25a21468ca71 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.592781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b1e37-0138-4c65-8ded-25a21468ca71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.595121 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b1e37-0138-4c65-8ded-25a21468ca71 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.610093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 911/1805] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/users => generated 287 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:27.613704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4da8aa-88c1-4af1-bfc1-c134f5548c19 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.625433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 895/1806] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:27.626485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4da8aa-88c1-4af1-bfc1-c134f5548c19 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.627002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b1e37-0138-4c65-8ded-25a21468ca71 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.631930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12f60a-c990-40f6-9848-63e875b2dc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.632692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12f60a-c990-40f6-9848-63e875b2dc1c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.633053 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12f60a-c990-40f6-9848-63e875b2dc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.633409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a12f60a-c990-40f6-9848-63e875b2dc1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.634271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12f60a-c990-40f6-9848-63e875b2dc1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.635948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a12f60a-c990-40f6-9848-63e875b2dc1c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.638191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b1e37-0138-4c65-8ded-25a21468ca71 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.642899 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92f80b-9768-474f-9e23-afc61d4934ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.643569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92f80b-9768-474f-9e23-afc61d4934ff admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.643862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92f80b-9768-474f-9e23-afc61d4934ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.644143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92f80b-9768-474f-9e23-afc61d4934ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.644877 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92f80b-9768-474f-9e23-afc61d4934ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.646506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92f80b-9768-474f-9e23-afc61d4934ff admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.655613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 912/1807] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:27.656806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12f60a-c990-40f6-9848-63e875b2dc1c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.664443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 896/1808] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:27.667273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92f80b-9768-474f-9e23-afc61d4934ff admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.672506 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a12f60a-c990-40f6-9848-63e875b2dc1c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.680975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92f80b-9768-474f-9e23-afc61d4934ff admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.681539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dee731-cb75-45d6-a30d-8ad8c2e397ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.682535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dee731-cb75-45d6-a30d-8ad8c2e397ca admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.682535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dee731-cb75-45d6-a30d-8ad8c2e397ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.682867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dee731-cb75-45d6-a30d-8ad8c2e397ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.683596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dee731-cb75-45d6-a30d-8ad8c2e397ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.685398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dee731-cb75-45d6-a30d-8ad8c2e397ca admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.686199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd5b08-4b9e-4d41-b32d-54bd029afe93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.687404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd5b08-4b9e-4d41-b32d-54bd029afe93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.688737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd5b08-4b9e-4d41-b32d-54bd029afe93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.689031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd5b08-4b9e-4d41-b32d-54bd029afe93 admin admin] PATH_INFO: `/v3/projects/b36ff036331e4e6fac501c341a9476ca/users/33cda0c5d1b04203bf7029add7b32171/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.700648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 913/1809] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:27.701756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dee731-cb75-45d6-a30d-8ad8c2e397ca admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.709757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd5b08-4b9e-4d41-b32d-54bd029afe93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36ff036331e4e6fac501c341a9476ca, user_id=33cda0c5d1b04203bf7029add7b32171, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.712087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dee731-cb75-45d6-a30d-8ad8c2e397ca admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.713466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd5b08-4b9e-4d41-b32d-54bd029afe93 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.716879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abc07e-659f-4b40-bf95-f0d7366f2690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.717808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abc07e-659f-4b40-bf95-f0d7366f2690 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.718123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abc07e-659f-4b40-bf95-f0d7366f2690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.718666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abc07e-659f-4b40-bf95-f0d7366f2690 admin admin] PATH_INFO: `/v3/projects/8fd9546078f04b8a83a9d92914082a76/users/87d36dd298994724945d46a9889b2502/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.728766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 897/1810] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:27 2022] PUT /identity/v3/projects/b36ff036331e4e6fac501c341a9476ca/users/33cda0c5d1b04203bf7029add7b32171/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:27.732936 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd5b08-4b9e-4d41-b32d-54bd029afe93 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.733509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abc07e-659f-4b40-bf95-f0d7366f2690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd9546078f04b8a83a9d92914082a76, user_id=87d36dd298994724945d46a9889b2502, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.736701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abc07e-659f-4b40-bf95-f0d7366f2690 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.750995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 914/1811] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:27 2022] PUT /identity/v3/projects/8fd9546078f04b8a83a9d92914082a76/users/87d36dd298994724945d46a9889b2502/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:27.762334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86796ff-dd25-47a2-acd8-547c0d939c21 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.762800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86796ff-dd25-47a2-acd8-547c0d939c21 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.763083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86796ff-dd25-47a2-acd8-547c0d939c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.789828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a86796ff-dd25-47a2-acd8-547c0d939c21 None None] MFA Rules not processed for user `87d36dd298994724945d46a9889b2502`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:27.791078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a86796ff-dd25-47a2-acd8-547c0d939c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:27.802208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd5b08-4b9e-4d41-b32d-54bd029afe93 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.864378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09516e21-273d-4709-8fe0-06cd300ab6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.865217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09516e21-273d-4709-8fe0-06cd300ab6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.865518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09516e21-273d-4709-8fe0-06cd300ab6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.865803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09516e21-273d-4709-8fe0-06cd300ab6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.866576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09516e21-273d-4709-8fe0-06cd300ab6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.868209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09516e21-273d-4709-8fe0-06cd300ab6b1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.878705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 915/1812] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:27.882351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 898/1813] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:27 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:27.883266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09516e21-273d-4709-8fe0-06cd300ab6b1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.883394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86796ff-dd25-47a2-acd8-547c0d939c21 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.893586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09516e21-273d-4709-8fe0-06cd300ab6b1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.894194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86796ff-dd25-47a2-acd8-547c0d939c21 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:27.898225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94637414-0965-4420-9b62-78b8b72d7999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.898888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94637414-0965-4420-9b62-78b8b72d7999 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.898991 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50352a40-f69b-42d2-9ef8-62271bd9c0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:27.899186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94637414-0965-4420-9b62-78b8b72d7999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.899458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94637414-0965-4420-9b62-78b8b72d7999 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.899994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352a40-f69b-42d2-9ef8-62271bd9c0da admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.900149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94637414-0965-4420-9b62-78b8b72d7999 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.900284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352a40-f69b-42d2-9ef8-62271bd9c0da admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.900549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50352a40-f69b-42d2-9ef8-62271bd9c0da admin admin] PATH_INFO: `/v3/projects/b36ff036331e4e6fac501c341a9476ca/users/33cda0c5d1b04203bf7029add7b32171/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.902306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94637414-0965-4420-9b62-78b8b72d7999 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.902621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50352a40-f69b-42d2-9ef8-62271bd9c0da admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36ff036331e4e6fac501c341a9476ca, user_id=33cda0c5d1b04203bf7029add7b32171, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:27.905223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50352a40-f69b-42d2-9ef8-62271bd9c0da admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:27.912114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:27.919048 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 916/1814] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:27 2022] PUT /identity/v3/projects/b36ff036331e4e6fac501c341a9476ca/users/33cda0c5d1b04203bf7029add7b32171/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:27.931009 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140c15e-47a0-47a7-98e0-5166990d532b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:27.931402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140c15e-47a0-47a7-98e0-5166990d532b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:27.931756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2140c15e-47a0-47a7-98e0-5166990d532b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:27.936385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 899/1815] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:27.940476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94637414-0965-4420-9b62-78b8b72d7999 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:27.969826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2140c15e-47a0-47a7-98e0-5166990d532b None None] MFA Rules not processed for user `33cda0c5d1b04203bf7029add7b32171`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:27.969826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2140c15e-47a0-47a7-98e0-5166990d532b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:27.988476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:28.007283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94637414-0965-4420-9b62-78b8b72d7999 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:28.011853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfd1de-7251-41d5-a25f-858cff295a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:28.012526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfd1de-7251-41d5-a25f-858cff295a08 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:28.012827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfd1de-7251-41d5-a25f-858cff295a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:28.013088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dfd1de-7251-41d5-a25f-858cff295a08 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:28.013799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfd1de-7251-41d5-a25f-858cff295a08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:28.015828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dfd1de-7251-41d5-a25f-858cff295a08 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:28.036981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 900/1816] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:28.041085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfd1de-7251-41d5-a25f-858cff295a08 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:28.045310 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 917/1817] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:27 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:28.052736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dfd1de-7251-41d5-a25f-858cff295a08 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:28.056755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140c15e-47a0-47a7-98e0-5166990d532b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:28.057708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3eb8d-e66a-4e5e-9d17-b7027383fcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:28.058113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2140c15e-47a0-47a7-98e0-5166990d532b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:28.058418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3eb8d-e66a-4e5e-9d17-b7027383fcb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:28.058709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3eb8d-e66a-4e5e-9d17-b7027383fcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:28.058988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3eb8d-e66a-4e5e-9d17-b7027383fcb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:28.059721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3eb8d-e66a-4e5e-9d17-b7027383fcb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:28.061346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3eb8d-e66a-4e5e-9d17-b7027383fcb7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:28.072312 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 901/1818] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:28.076203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3eb8d-e66a-4e5e-9d17-b7027383fcb7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:28.085615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3eb8d-e66a-4e5e-9d17-b7027383fcb7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:28.090368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0851d-e9ca-4e4f-9596-748b7148a5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:28.091233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0851d-e9ca-4e4f-9596-748b7148a5f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:28.091520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0851d-e9ca-4e4f-9596-748b7148a5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:28.091796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0851d-e9ca-4e4f-9596-748b7148a5f1 admin admin] PATH_INFO: `/v3/projects/c227410912a74bfb9d598cd5e4de075a/users/08966f0ea3d54e8eb72410be4fb6976b/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:28.106381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0851d-e9ca-4e4f-9596-748b7148a5f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c227410912a74bfb9d598cd5e4de075a, user_id=08966f0ea3d54e8eb72410be4fb6976b, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:28.109008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc0851d-e9ca-4e4f-9596-748b7148a5f1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:28.115208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2140c15e-47a0-47a7-98e0-5166990d532b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:28.119754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d3779-ce9e-4040-abac-2d0f5bbbe709 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:28.121952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3779-ce9e-4040-abac-2d0f5bbbe709 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:28.122304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3779-ce9e-4040-abac-2d0f5bbbe709 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:28.122712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 902/1819] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:28 2022] PUT /identity/v3/projects/c227410912a74bfb9d598cd5e4de075a/users/08966f0ea3d54e8eb72410be4fb6976b/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:28.122832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3779-ce9e-4040-abac-2d0f5bbbe709 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:28.126160 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0851d-e9ca-4e4f-9596-748b7148a5f1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:28.132796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d3779-ce9e-4040-abac-2d0f5bbbe709 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:28.134259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d3779-ce9e-4040-abac-2d0f5bbbe709 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:28.144638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 918/1820] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:28 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:28.193493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc0851d-e9ca-4e4f-9596-748b7148a5f1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:28.198654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46bd6-e1f8-40cb-a30b-a28085fc26e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:28.199360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46bd6-e1f8-40cb-a30b-a28085fc26e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:28.199644 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46bd6-e1f8-40cb-a30b-a28085fc26e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:28.199921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff46bd6-e1f8-40cb-a30b-a28085fc26e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:28.200649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff46bd6-e1f8-40cb-a30b-a28085fc26e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:28.202314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff46bd6-e1f8-40cb-a30b-a28085fc26e7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:28.214437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 903/1821] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:28.219449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d3779-ce9e-4040-abac-2d0f5bbbe709 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:28.230353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d3779-ce9e-4040-abac-2d0f5bbbe709 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:28.235660 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2357857d-5714-4cf5-96af-8afba57081df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:28.236566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357857d-5714-4cf5-96af-8afba57081df admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:28.236973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357857d-5714-4cf5-96af-8afba57081df admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:28.237255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2357857d-5714-4cf5-96af-8afba57081df admin admin] PATH_INFO: `/v3/projects/c227410912a74bfb9d598cd5e4de075a/users/08966f0ea3d54e8eb72410be4fb6976b/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:28.239275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357857d-5714-4cf5-96af-8afba57081df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c227410912a74bfb9d598cd5e4de075a, user_id=08966f0ea3d54e8eb72410be4fb6976b, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:28.242220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2357857d-5714-4cf5-96af-8afba57081df admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:28.246958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46bd6-e1f8-40cb-a30b-a28085fc26e7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:28.253930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 919/1822] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:28 2022] PUT /identity/v3/projects/c227410912a74bfb9d598cd5e4de075a/users/08966f0ea3d54e8eb72410be4fb6976b/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:28.263938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebece96-f4a5-47ba-8950-3320a25bf2fd None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:28.264234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebece96-f4a5-47ba-8950-3320a25bf2fd None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:28.264536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebece96-f4a5-47ba-8950-3320a25bf2fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:28.290637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4ebece96-f4a5-47ba-8950-3320a25bf2fd None None] MFA Rules not processed for user `08966f0ea3d54e8eb72410be4fb6976b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:28.291944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4ebece96-f4a5-47ba-8950-3320a25bf2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:28.294188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:28.308101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff46bd6-e1f8-40cb-a30b-a28085fc26e7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:28.357793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95113fa-d87b-4149-bec1-2f95d562875b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:28.358477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95113fa-d87b-4149-bec1-2f95d562875b service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:28.358765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95113fa-d87b-4149-bec1-2f95d562875b service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:28.359066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95113fa-d87b-4149-bec1-2f95d562875b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:28.360276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 920/1823] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:28 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:28.371279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebece96-f4a5-47ba-8950-3320a25bf2fd None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:28.372617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4ebece96-f4a5-47ba-8950-3320a25bf2fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:28.384978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebece96-f4a5-47ba-8950-3320a25bf2fd None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:28.389021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf71902-85a2-4f55-8745-68ee184144a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:28.389663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf71902-85a2-4f55-8745-68ee184144a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:28.389947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf71902-85a2-4f55-8745-68ee184144a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:28.390243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf71902-85a2-4f55-8745-68ee184144a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:28.423117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:28.426823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95113fa-d87b-4149-bec1-2f95d562875b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:28.430418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95113fa-d87b-4149-bec1-2f95d562875b service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:28.442889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 904/1824] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:28.463333 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf71902-85a2-4f55-8745-68ee184144a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:28.465403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf71902-85a2-4f55-8745-68ee184144a6 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:28.475072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 921/1825] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:34.335971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95113fa-d87b-4149-bec1-2f95d562875b service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:34.369898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf71902-85a2-4f55-8745-68ee184144a6 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:34.444705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95113fa-d87b-4149-bec1-2f95d562875b service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:34.450230 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:34.451500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:34.451888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:34.452275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] PATH_INFO: `/v3/users/9e9111d750a24e77b461488fef77e3ee` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:34.462436 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e9111d750a24e77b461488fef77e3ee)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:34.462436 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:34.481245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf71902-85a2-4f55-8745-68ee184144a6 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:34.486962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:34.489344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:34.489731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:34.490107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] PATH_INFO: `/v3/users/87d36dd298994724945d46a9889b2502` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:34.491426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d36dd298994724945d46a9889b2502)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:34.493526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:34.529438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e9111d750a24e77b461488fef77e3ee'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:34.530289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e9111d750a24e77b461488fef77e3ee'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:34.531836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e9111d750a24e77b461488fef77e3ee'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:34.536792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:34.555868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e9111d750a24e77b461488fef77e3ee'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:34.565907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aa208d0a-5d51-49cc-839f-5610f5bb38c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e9111d750a24e77b461488fef77e3ee'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:34.577905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d36dd298994724945d46a9889b2502'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:34.579183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d36dd298994724945d46a9889b2502'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:34.585365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 905/1826] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:34 2022] DELETE /identity/v3/users/9e9111d750a24e77b461488fef77e3ee => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:34.590083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d36dd298994724945d46a9889b2502'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:34.590786 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00db29-bfe1-4791-9ab7-e2fb20cdb4d3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:34.591178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00db29-bfe1-4791-9ab7-e2fb20cdb4d3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:34.591581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00db29-bfe1-4791-9ab7-e2fb20cdb4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:34.600339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d36dd298994724945d46a9889b2502'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:34.601690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-e66fe7e4-d0a5-48b3-a0f9-7c32deac8e7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d36dd298994724945d46a9889b2502'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:34.614806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dc00db29-bfe1-4791-9ab7-e2fb20cdb4d3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:34.616429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dc00db29-bfe1-4791-9ab7-e2fb20cdb4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:34.626988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 922/1827] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:34 2022] DELETE /identity/v3/users/87d36dd298994724945d46a9889b2502 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:34.631685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891631b-3a90-4267-b0ba-f4a94a90f45f None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:34.631978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891631b-3a90-4267-b0ba-f4a94a90f45f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:34.632263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891631b-3a90-4267-b0ba-f4a94a90f45f None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:34.633721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 923/1828] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:05:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:05:34.634727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7f71d-bfa2-4512-86cf-c6355be99849 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:34.635140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7f71d-bfa2-4512-86cf-c6355be99849 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:34.636154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb7f71d-bfa2-4512-86cf-c6355be99849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:34.654943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0fb7f71d-bfa2-4512-86cf-c6355be99849 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:34.656766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0fb7f71d-bfa2-4512-86cf-c6355be99849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:34.699111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 906/1829] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:34.707555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c58f3-c981-4974-8527-3268bcadf2e6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:34.707853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c58f3-c981-4974-8527-3268bcadf2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:34.708137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c58f3-c981-4974-8527-3268bcadf2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:34.715997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:34.731816 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0f3c58f3-c981-4974-8527-3268bcadf2e6 None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:34.733383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0f3c58f3-c981-4974-8527-3268bcadf2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:34.742891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 924/1830] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:34.743981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7f71d-bfa2-4512-86cf-c6355be99849 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:34.745661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0fb7f71d-bfa2-4512-86cf-c6355be99849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:34.811680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb7f71d-bfa2-4512-86cf-c6355be99849 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:34.813514 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 907/1831] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:05:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:34.814549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c58f3-c981-4974-8527-3268bcadf2e6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:34.815858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0f3c58f3-c981-4974-8527-3268bcadf2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:34.816407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e918d6-2028-4d65-9339-58cce0cb779b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:34.817131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e918d6-2028-4d65-9339-58cce0cb779b service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:34.817482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e918d6-2028-4d65-9339-58cce0cb779b service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:34.817765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e918d6-2028-4d65-9339-58cce0cb779b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:34.829218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c58f3-c981-4974-8527-3268bcadf2e6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:34.833390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556bcf2-1893-40fd-8e76-2cd05b0773b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:34.834071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556bcf2-1893-40fd-8e76-2cd05b0773b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:34.834359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556bcf2-1893-40fd-8e76-2cd05b0773b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:34.834688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c556bcf2-1893-40fd-8e76-2cd05b0773b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:34.846294 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556bcf2-1893-40fd-8e76-2cd05b0773b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:34.848247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c556bcf2-1893-40fd-8e76-2cd05b0773b5 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:34.851771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e918d6-2028-4d65-9339-58cce0cb779b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:34.854752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e918d6-2028-4d65-9339-58cce0cb779b service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:34.861911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 908/1832] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:34.863382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556bcf2-1893-40fd-8e76-2cd05b0773b5 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:34.864996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c556bcf2-1893-40fd-8e76-2cd05b0773b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:34.866918 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 925/1833] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:34.884863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c556bcf2-1893-40fd-8e76-2cd05b0773b5 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:34.891079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd1938-5e2d-4f8c-8257-873a23293c83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:34.891736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd1938-5e2d-4f8c-8257-873a23293c83 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:34.892018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd1938-5e2d-4f8c-8257-873a23293c83 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:34.892295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd1938-5e2d-4f8c-8257-873a23293c83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:34.901554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd1938-5e2d-4f8c-8257-873a23293c83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:34.903168 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd1938-5e2d-4f8c-8257-873a23293c83 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:34.913869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 909/1834] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:35.236038 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e918d6-2028-4d65-9339-58cce0cb779b service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:35.237548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f4e918d6-2028-4d65-9339-58cce0cb779b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:35.256477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e918d6-2028-4d65-9339-58cce0cb779b service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:35.261611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:35.262818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:35.263330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:35.263719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] PATH_INFO: `/v3/projects/8fd9546078f04b8a83a9d92914082a76` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:35.265332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd9546078f04b8a83a9d92914082a76)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:35.268143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:35.335308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd9546078f04b8a83a9d92914082a76', 'request_id': 'req-5464304b-92a6-4351-a637-285faeae444d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.341276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd9546078f04b8a83a9d92914082a76', 'request_id': 'req-5464304b-92a6-4351-a637-285faeae444d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.350265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd9546078f04b8a83a9d92914082a76', 'request_id': 'req-5464304b-92a6-4351-a637-285faeae444d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.365101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] The token cache is being invalidate because project 8fd9546078f04b8a83a9d92914082a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:35.365398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.366380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.368164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 926/1835] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:35 2022] DELETE /identity/v3/projects/8fd9546078f04b8a83a9d92914082a76 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:35.372809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd1938-5e2d-4f8c-8257-873a23293c83 service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:35.385253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd1938-5e2d-4f8c-8257-873a23293c83 service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:35.430977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:35.463515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:35.464534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:35.465114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:35.465408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] PATH_INFO: `/v3/users/08966f0ea3d54e8eb72410be4fb6976b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:35.466722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08966f0ea3d54e8eb72410be4fb6976b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:35.469202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:35.524001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08966f0ea3d54e8eb72410be4fb6976b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.525483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08966f0ea3d54e8eb72410be4fb6976b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.526153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08966f0ea3d54e8eb72410be4fb6976b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.546380 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08966f0ea3d54e8eb72410be4fb6976b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.569405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-371ed3a4-4aa8-4e00-9e88-c2378760b169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08966f0ea3d54e8eb72410be4fb6976b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.573220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:35.575267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:35.592413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464304b-92a6-4351-a637-285faeae444d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:35.596058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 910/1836] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:35 2022] DELETE /identity/v3/users/08966f0ea3d54e8eb72410be4fb6976b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:35.682418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:35.683421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:35.683813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:35.684099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] PATH_INFO: `/v3/users/0906cefead324b768df7246459704500` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:35.686008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0906cefead324b768df7246459704500)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:35.689026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:35.711066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:35.760391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0906cefead324b768df7246459704500'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.761490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0906cefead324b768df7246459704500'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.818724 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0906cefead324b768df7246459704500'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.835890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0906cefead324b768df7246459704500'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.836627 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0906cefead324b768df7246459704500'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:35.866453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 927/1837] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:35 2022] DELETE /identity/v3/users/0906cefead324b768df7246459704500 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:35.871934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7293504-b4a0-479a-bf48-807d49da3d3c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:35.872217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7293504-b4a0-479a-bf48-807d49da3d3c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:35.872503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7293504-b4a0-479a-bf48-807d49da3d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:35.895171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e7293504-b4a0-479a-bf48-807d49da3d3c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:35.896811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e7293504-b4a0-479a-bf48-807d49da3d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:36.108290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:36.110847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:36.139565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 911/1838] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:36.151544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7293504-b4a0-479a-bf48-807d49da3d3c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:36.154839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e7293504-b4a0-479a-bf48-807d49da3d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:36.227546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb935675-994a-4eb7-9b6a-ee3049dc5851 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:36.232977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:36.234130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:36.234924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:36.235206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] PATH_INFO: `/v3/projects/1c66b7f14f9c44019c579e57913cc988` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:36.236396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c66b7f14f9c44019c579e57913cc988)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:36.238425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:36.244554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7293504-b4a0-479a-bf48-807d49da3d3c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:36.249219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbf9fd-411b-4628-9c23-8e21c191cef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:36.249917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbf9fd-411b-4628-9c23-8e21c191cef5 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:36.250361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbf9fd-411b-4628-9c23-8e21c191cef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:36.250656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bbf9fd-411b-4628-9c23-8e21c191cef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:36.262822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbf9fd-411b-4628-9c23-8e21c191cef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:36.264567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bbf9fd-411b-4628-9c23-8e21c191cef5 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:36.268983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:36.276607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 912/1839] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:36.277586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbf9fd-411b-4628-9c23-8e21c191cef5 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:36.278993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d4bbf9fd-411b-4628-9c23-8e21c191cef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:36.300438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bbf9fd-411b-4628-9c23-8e21c191cef5 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:36.307640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c66b7f14f9c44019c579e57913cc988', 'request_id': 'req-8eaf3210-7f66-453f-acef-418e0a20ac43'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.314397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c66b7f14f9c44019c579e57913cc988', 'request_id': 'req-8eaf3210-7f66-453f-acef-418e0a20ac43'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.325026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c66b7f14f9c44019c579e57913cc988', 'request_id': 'req-8eaf3210-7f66-453f-acef-418e0a20ac43'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.346253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] The token cache is being invalidate because project 1c66b7f14f9c44019c579e57913cc988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:36.346598 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.347301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.349182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 928/1840] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:36 2022] DELETE /identity/v3/projects/1c66b7f14f9c44019c579e57913cc988 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:36.352922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:36.354688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:36.371854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:36.372882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:36.373452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:36.373561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] PATH_INFO: `/v3/projects/c227410912a74bfb9d598cd5e4de075a` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:36.374991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c227410912a74bfb9d598cd5e4de075a)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:36.377012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:36.393416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:36.435140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c227410912a74bfb9d598cd5e4de075a', 'request_id': 'req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.439207 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaf3210-7f66-453f-acef-418e0a20ac43 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:36.451475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c227410912a74bfb9d598cd5e4de075a', 'request_id': 'req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.458966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c227410912a74bfb9d598cd5e4de075a', 'request_id': 'req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.472947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] The token cache is being invalidate because project c227410912a74bfb9d598cd5e4de075a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:36.473256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.473934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-abbbe9f0-e325-4c1b-97e9-456af8d4b7d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.475848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 913/1841] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:36 2022] DELETE /identity/v3/projects/c227410912a74bfb9d598cd5e4de075a => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:36.480542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1247f-3a70-4f83-a866-eebbab60a5a3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:36.480840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1247f-3a70-4f83-a866-eebbab60a5a3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:36.481757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf1247f-3a70-4f83-a866-eebbab60a5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:36.503479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-bdf1247f-3a70-4f83-a866-eebbab60a5a3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:36.504895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bdf1247f-3a70-4f83-a866-eebbab60a5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:36.512841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:36.513988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:36.514498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:36.514931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] PATH_INFO: `/v3/users/33cda0c5d1b04203bf7029add7b32171` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:36.516934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] RBAC: Authorizing `identity:delete_user(user_id=33cda0c5d1b04203bf7029add7b32171)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:36.518940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:36.591541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 914/1842] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:36.592184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33cda0c5d1b04203bf7029add7b32171'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.593057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33cda0c5d1b04203bf7029add7b32171'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.600522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1247f-3a70-4f83-a866-eebbab60a5a3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:36.602886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bdf1247f-3a70-4f83-a866-eebbab60a5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:36.610470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33cda0c5d1b04203bf7029add7b32171'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.615437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf1247f-3a70-4f83-a866-eebbab60a5a3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:36.621918 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:36.622523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33cda0c5d1b04203bf7029add7b32171'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.622894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:36.623207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:36.623566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] PATH_INFO: `/v3/projects/5fafb0ab957a4f069ef1cfea7eaa5e0e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:36.623851 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4c02fae5-5a77-4292-bb02-f4456869dfed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33cda0c5d1b04203bf7029add7b32171'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.625388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fafb0ab957a4f069ef1cfea7eaa5e0e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:36.627794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:36.649151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 929/1843] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:36 2022] DELETE /identity/v3/users/33cda0c5d1b04203bf7029add7b32171 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:36.652206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c1800-c6f7-465e-b85d-2474e7b1fa26 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:36.652206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c1800-c6f7-465e-b85d-2474e7b1fa26 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:36.652206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c1800-c6f7-465e-b85d-2474e7b1fa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:36.657036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:36.678128 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-be9c1800-c6f7-465e-b85d-2474e7b1fa26 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:36.679909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-be9c1800-c6f7-465e-b85d-2474e7b1fa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:36.697165 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fafb0ab957a4f069ef1cfea7eaa5e0e', 'request_id': 'req-5908e6ff-6384-45e7-b128-c844607999f1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.708000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fafb0ab957a4f069ef1cfea7eaa5e0e', 'request_id': 'req-5908e6ff-6384-45e7-b128-c844607999f1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.715570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fafb0ab957a4f069ef1cfea7eaa5e0e', 'request_id': 'req-5908e6ff-6384-45e7-b128-c844607999f1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.729919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] The token cache is being invalidate because project 5fafb0ab957a4f069ef1cfea7eaa5e0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:36.730781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.731531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.734061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 915/1844] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:36 2022] DELETE /identity/v3/projects/5fafb0ab957a4f069ef1cfea7eaa5e0e => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:36.737646 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:36.738841 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:36.741701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:36.759411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 930/1845] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:36.763367 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c1800-c6f7-465e-b85d-2474e7b1fa26 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:36.773675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c1800-c6f7-465e-b85d-2474e7b1fa26 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:36.778823 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5a9f-4642-4863-bb6d-eeb3b737b4f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:36.779484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5a9f-4642-4863-bb6d-eeb3b737b4f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:36.779983 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5a9f-4642-4863-bb6d-eeb3b737b4f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:36.780190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a5a9f-4642-4863-bb6d-eeb3b737b4f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:36.781180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a5a9f-4642-4863-bb6d-eeb3b737b4f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:36.781566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a5a9f-4642-4863-bb6d-eeb3b737b4f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:36.783907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a5a9f-4642-4863-bb6d-eeb3b737b4f2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:36.800392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 931/1846] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:05:36.804955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5a9f-4642-4863-bb6d-eeb3b737b4f2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:36.818517 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a5a9f-4642-4863-bb6d-eeb3b737b4f2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:36.823951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21e368-1b74-42b9-9465-1ca83a0656a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:36.824715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21e368-1b74-42b9-9465-1ca83a0656a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:36.825001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21e368-1b74-42b9-9465-1ca83a0656a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:36.825276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21e368-1b74-42b9-9465-1ca83a0656a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:36.826441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21e368-1b74-42b9-9465-1ca83a0656a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:36.829184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc21e368-1b74-42b9-9465-1ca83a0656a9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:36.829600 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5908e6ff-6384-45e7-b128-c844607999f1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:36.843211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:36.844378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:36.846826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:36.847220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] PATH_INFO: `/v3/users/bb6338d89ee3406094136ba08161623f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:36.850182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb6338d89ee3406094136ba08161623f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:36.852199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:36.878489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 932/1847] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:36 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:36.882423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21e368-1b74-42b9-9465-1ca83a0656a9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:36.960012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb6338d89ee3406094136ba08161623f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.960743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb6338d89ee3406094136ba08161623f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.961984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb6338d89ee3406094136ba08161623f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.989907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb6338d89ee3406094136ba08161623f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:36.992368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21e368-1b74-42b9-9465-1ca83a0656a9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:37.001083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1226274-9e98-47cb-b2d6-caf6969f4b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:37.001883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1226274-9e98-47cb-b2d6-caf6969f4b0d admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.003538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb6338d89ee3406094136ba08161623f'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:37.006356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1226274-9e98-47cb-b2d6-caf6969f4b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.007242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1226274-9e98-47cb-b2d6-caf6969f4b0d admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.009068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1226274-9e98-47cb-b2d6-caf6969f4b0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:37.013547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1226274-9e98-47cb-b2d6-caf6969f4b0d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:37.047472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 933/1848] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:36 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:37.049388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 916/1849] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:36 2022] DELETE /identity/v3/users/bb6338d89ee3406094136ba08161623f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:37.055288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1226274-9e98-47cb-b2d6-caf6969f4b0d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:37.057633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a1226274-9e98-47cb-b2d6-caf6969f4b0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:37.076300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1226274-9e98-47cb-b2d6-caf6969f4b0d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:37.150123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdbd76-c414-4d00-8f4b-f1f347312658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:37.150863 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbd76-c414-4d00-8f4b-f1f347312658 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.151241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbd76-c414-4d00-8f4b-f1f347312658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.151518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdbd76-c414-4d00-8f4b-f1f347312658 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.152464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdbd76-c414-4d00-8f4b-f1f347312658 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:37.154878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdbd76-c414-4d00-8f4b-f1f347312658 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:37.189700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 934/1850] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:37.192765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:37.204986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8520658d-5e58-4380-960d-ca34fdecfd8a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:37.209802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05419ede-5321-494e-886a-a95e9ff180c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:37.210535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05419ede-5321-494e-886a-a95e9ff180c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.210825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05419ede-5321-494e-886a-a95e9ff180c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.211108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05419ede-5321-494e-886a-a95e9ff180c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.211844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05419ede-5321-494e-886a-a95e9ff180c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:37.213460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05419ede-5321-494e-886a-a95e9ff180c9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:37.228134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 917/1851] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:37.229078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdbd76-c414-4d00-8f4b-f1f347312658 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:37.240389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdbd76-c414-4d00-8f4b-f1f347312658 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:37.245191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8e13a-0008-4b22-bbb9-b2d97027cc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:37.246130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8e13a-0008-4b22-bbb9-b2d97027cc11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.246419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8e13a-0008-4b22-bbb9-b2d97027cc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.246708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8e13a-0008-4b22-bbb9-b2d97027cc11 admin admin] PATH_INFO: `/v3/projects/90f7307168c24bd1aef024ca8984c16e/users/f45e05b1ceb34bbbbf79fee531174707/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.264558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8e13a-0008-4b22-bbb9-b2d97027cc11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90f7307168c24bd1aef024ca8984c16e, user_id=f45e05b1ceb34bbbbf79fee531174707, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:37.267373 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca8e13a-0008-4b22-bbb9-b2d97027cc11 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:37.284071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 935/1852] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:37 2022] PUT /identity/v3/projects/90f7307168c24bd1aef024ca8984c16e/users/f45e05b1ceb34bbbbf79fee531174707/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:37.287359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05419ede-5321-494e-886a-a95e9ff180c9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:37.290954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-05419ede-5321-494e-886a-a95e9ff180c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:37.294075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c2422-0a78-4059-b961-0b96475eb737 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.294075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c2422-0a78-4059-b961-0b96475eb737 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.294288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c2422-0a78-4059-b961-0b96475eb737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.310843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05419ede-5321-494e-886a-a95e9ff180c9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:37.337195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:37.353257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-fd9c2422-0a78-4059-b961-0b96475eb737 None None] MFA Rules not processed for user `f45e05b1ceb34bbbbf79fee531174707`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:37.355924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fd9c2422-0a78-4059-b961-0b96475eb737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:37.367153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:37.417822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:37.420608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.421085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.422919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] PATH_INFO: `/v3/projects/b36ff036331e4e6fac501c341a9476ca` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.427089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36ff036331e4e6fac501c341a9476ca)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:37.429188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:37.480318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 936/1853] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:37.481335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c2422-0a78-4059-b961-0b96475eb737 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:37.494070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c2422-0a78-4059-b961-0b96475eb737 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:37.519544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36ff036331e4e6fac501c341a9476ca', 'request_id': 'req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:37.535495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36ff036331e4e6fac501c341a9476ca', 'request_id': 'req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:37.549551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36ff036331e4e6fac501c341a9476ca', 'request_id': 'req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:37.569272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8308b-bdb7-47b7-92a3-a21faf751554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:37.570514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8308b-bdb7-47b7-92a3-a21faf751554 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.570635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8308b-bdb7-47b7-92a3-a21faf751554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.571066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8308b-bdb7-47b7-92a3-a21faf751554 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.572212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8308b-bdb7-47b7-92a3-a21faf751554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:37.574908 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8308b-bdb7-47b7-92a3-a21faf751554 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:37.598330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] The token cache is being invalidate because project b36ff036331e4e6fac501c341a9476ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:37.598708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:37.600714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea347e2c-0177-46ca-8b1d-f5ee994ccbf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:37.606739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 918/1854] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:37 2022] DELETE /identity/v3/projects/b36ff036331e4e6fac501c341a9476ca => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:37.613003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba23709-f923-4120-9753-9b001c7efcbe None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.613211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba23709-f923-4120-9753-9b001c7efcbe None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.614102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba23709-f923-4120-9753-9b001c7efcbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.662265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 937/1855] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:37 2022] POST /identity/v3/projects => generated 377 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:37.672419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8308b-bdb7-47b7-92a3-a21faf751554 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:37.678487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-45d8308b-bdb7-47b7-92a3-a21faf751554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:37.715715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-eba23709-f923-4120-9753-9b001c7efcbe None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:37.726268 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-eba23709-f923-4120-9753-9b001c7efcbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:37.905888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 919/1856] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:37.913967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95961bd-3058-4046-888d-96ff1fdf32d9 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.914377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95961bd-3058-4046-888d-96ff1fdf32d9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.914736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95961bd-3058-4046-888d-96ff1fdf32d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.939112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8308b-bdb7-47b7-92a3-a21faf751554 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:37.944245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b95961bd-3058-4046-888d-96ff1fdf32d9 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:37.945947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b95961bd-3058-4046-888d-96ff1fdf32d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:37.958447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 920/1857] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:37.962773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95961bd-3058-4046-888d-96ff1fdf32d9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:37.968450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6711-4448-4396-8839-c9e46f1239a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:37.970702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6711-4448-4396-8839-c9e46f1239a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.970987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6711-4448-4396-8839-c9e46f1239a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.971273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bb6711-4448-4396-8839-c9e46f1239a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.971977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb6711-4448-4396-8839-c9e46f1239a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:37.973056 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95961bd-3058-4046-888d-96ff1fdf32d9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:37.975646 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bb6711-4448-4396-8839-c9e46f1239a1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:37.978761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d2c5e-178c-4ccb-8884-76e1eadeeae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:37.979652 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d2c5e-178c-4ccb-8884-76e1eadeeae8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:37.980017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d2c5e-178c-4ccb-8884-76e1eadeeae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:37.980371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d2c5e-178c-4ccb-8884-76e1eadeeae8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:37.981186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d2c5e-178c-4ccb-8884-76e1eadeeae8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:37.981573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d2c5e-178c-4ccb-8884-76e1eadeeae8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:37.983306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0d2c5e-178c-4ccb-8884-76e1eadeeae8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.006079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 921/1858] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:05:38.008833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d2c5e-178c-4ccb-8884-76e1eadeeae8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.021801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 938/1859] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:37 2022] POST /identity/v3/users => generated 289 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:38.025215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6711-4448-4396-8839-c9e46f1239a1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.028170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d2c5e-178c-4ccb-8884-76e1eadeeae8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.038242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29247dcf-6f35-4fb0-a6f1-31da6892afe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.039685 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29247dcf-6f35-4fb0-a6f1-31da6892afe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.040323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29247dcf-6f35-4fb0-a6f1-31da6892afe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.040452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29247dcf-6f35-4fb0-a6f1-31da6892afe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.041358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29247dcf-6f35-4fb0-a6f1-31da6892afe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.044399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bb6711-4448-4396-8839-c9e46f1239a1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.045771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29247dcf-6f35-4fb0-a6f1-31da6892afe1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.052118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c6b35-c5cc-4803-9870-e1f38c2d7375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.053034 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c6b35-c5cc-4803-9870-e1f38c2d7375 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.053988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c6b35-c5cc-4803-9870-e1f38c2d7375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.054602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c6b35-c5cc-4803-9870-e1f38c2d7375 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.055530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c6b35-c5cc-4803-9870-e1f38c2d7375 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.057778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c6b35-c5cc-4803-9870-e1f38c2d7375 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.078103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 939/1860] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:38.082107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c6b35-c5cc-4803-9870-e1f38c2d7375 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.098091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 922/1861] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:38 2022] POST /identity/v3/projects => generated 389 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:38.099249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c6b35-c5cc-4803-9870-e1f38c2d7375 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.099970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29247dcf-6f35-4fb0-a6f1-31da6892afe1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.122681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:38.136544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:38.230455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29247dcf-6f35-4fb0-a6f1-31da6892afe1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.236687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd2336-9db8-4065-af0e-ca2d9663aeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.237260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328a94b-24c7-4b1d-8331-53744ae4c28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.237627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd2336-9db8-4065-af0e-ca2d9663aeb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.238022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd2336-9db8-4065-af0e-ca2d9663aeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.238551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddd2336-9db8-4065-af0e-ca2d9663aeb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.238833 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328a94b-24c7-4b1d-8331-53744ae4c28d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.239521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd2336-9db8-4065-af0e-ca2d9663aeb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.239975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328a94b-24c7-4b1d-8331-53744ae4c28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.240410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328a94b-24c7-4b1d-8331-53744ae4c28d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.241321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328a94b-24c7-4b1d-8331-53744ae4c28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.242317 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddd2336-9db8-4065-af0e-ca2d9663aeb3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.243370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c328a94b-24c7-4b1d-8331-53744ae4c28d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.266180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 940/1862] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:38.271871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328a94b-24c7-4b1d-8331-53744ae4c28d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.275870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 923/1863] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:38 2022] POST /identity/v3/users => generated 295 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:38.277127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd2336-9db8-4065-af0e-ca2d9663aeb3 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.294260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddd2336-9db8-4065-af0e-ca2d9663aeb3 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.297490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328a94b-24c7-4b1d-8331-53744ae4c28d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.301332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ce06c-2320-4165-bc85-820e46b87f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.302482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ce06c-2320-4165-bc85-820e46b87f16 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.303188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ce06c-2320-4165-bc85-820e46b87f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.303555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248ce06c-2320-4165-bc85-820e46b87f16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.304824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ce06c-2320-4165-bc85-820e46b87f16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.306951 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248ce06c-2320-4165-bc85-820e46b87f16 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.307587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbb4785-9046-4e20-b7b1-0080942b8099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.309436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb4785-9046-4e20-b7b1-0080942b8099 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.310361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb4785-9046-4e20-b7b1-0080942b8099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.311666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbb4785-9046-4e20-b7b1-0080942b8099 admin admin] PATH_INFO: `/v3/projects/c627a339d26c4bf594ab064ba7df11d6/users/291496959b2747d7b62ebcca3f413511/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.322090 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 924/1864] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:38.324907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ce06c-2320-4165-bc85-820e46b87f16 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.339573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb4785-9046-4e20-b7b1-0080942b8099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c627a339d26c4bf594ab064ba7df11d6, user_id=291496959b2747d7b62ebcca3f413511, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.340512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248ce06c-2320-4165-bc85-820e46b87f16 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.342701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbb4785-9046-4e20-b7b1-0080942b8099 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.350488 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208bbd9-e0df-40cf-91ba-e52e493391a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.351176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208bbd9-e0df-40cf-91ba-e52e493391a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.351467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208bbd9-e0df-40cf-91ba-e52e493391a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.351743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208bbd9-e0df-40cf-91ba-e52e493391a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.352567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208bbd9-e0df-40cf-91ba-e52e493391a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.354302 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208bbd9-e0df-40cf-91ba-e52e493391a9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.371468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 925/1865] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:38.371818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 941/1866] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:38 2022] PUT /identity/v3/projects/c627a339d26c4bf594ab064ba7df11d6/users/291496959b2747d7b62ebcca3f413511/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:38.381629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b158ef-078b-442c-879f-b1a0e7900f6b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.381810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208bbd9-e0df-40cf-91ba-e52e493391a9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.381950 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b158ef-078b-442c-879f-b1a0e7900f6b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.382253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b158ef-078b-442c-879f-b1a0e7900f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.407179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208bbd9-e0df-40cf-91ba-e52e493391a9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.420718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2b158ef-078b-442c-879f-b1a0e7900f6b None None] MFA Rules not processed for user `291496959b2747d7b62ebcca3f413511`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:38.423114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2b158ef-078b-442c-879f-b1a0e7900f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:38.497958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6c48b-d92e-4979-bb86-28f8e8f5cb5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.500007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6c48b-d92e-4979-bb86-28f8e8f5cb5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.500434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6c48b-d92e-4979-bb86-28f8e8f5cb5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.500851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6c48b-d92e-4979-bb86-28f8e8f5cb5c admin admin] PATH_INFO: `/v3/projects/61aeeea6c3ab471f8c227b80a4be7fb0/users/7fd97262fc464caab3e1bc6252c77f25/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.519650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6c48b-d92e-4979-bb86-28f8e8f5cb5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=61aeeea6c3ab471f8c227b80a4be7fb0, user_id=7fd97262fc464caab3e1bc6252c77f25, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.525344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6c48b-d92e-4979-bb86-28f8e8f5cb5c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.558834 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 926/1867] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:38 2022] PUT /identity/v3/projects/61aeeea6c3ab471f8c227b80a4be7fb0/users/7fd97262fc464caab3e1bc6252c77f25/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:38.563985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d13bdd-4fd9-487d-ab33-a1de736d9abc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.563985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d13bdd-4fd9-487d-ab33-a1de736d9abc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.563985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d13bdd-4fd9-487d-ab33-a1de736d9abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.573863 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 942/1868] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:38.579402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b158ef-078b-442c-879f-b1a0e7900f6b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.604892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b158ef-078b-442c-879f-b1a0e7900f6b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.606300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-24d13bdd-4fd9-487d-ab33-a1de736d9abc None None] MFA Rules not processed for user `7fd97262fc464caab3e1bc6252c77f25`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:38.607953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-24d13bdd-4fd9-487d-ab33-a1de736d9abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:38.713277 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:38.724191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ada6a8-0635-414d-8cbb-3b2191b16366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.725402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ada6a8-0635-414d-8cbb-3b2191b16366 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.725689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ada6a8-0635-414d-8cbb-3b2191b16366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.725992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ada6a8-0635-414d-8cbb-3b2191b16366 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.726725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ada6a8-0635-414d-8cbb-3b2191b16366 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.728669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ada6a8-0635-414d-8cbb-3b2191b16366 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.748185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 927/1869] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:38.748185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d13bdd-4fd9-487d-ab33-a1de736d9abc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.748185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-24d13bdd-4fd9-487d-ab33-a1de736d9abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:38.765347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d13bdd-4fd9-487d-ab33-a1de736d9abc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.766662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:38.771746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.773663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.774075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.774391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] PATH_INFO: `/v3/projects/7065a845fd404bbfbf11c1ad766e8c09` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.776336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7065a845fd404bbfbf11c1ad766e8c09)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.778981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.780258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 943/1870] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:38 2022] POST /identity/v3/projects => generated 375 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:38.782221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ada6a8-0635-414d-8cbb-3b2191b16366 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.843649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7065a845fd404bbfbf11c1ad766e8c09', 'request_id': 'req-58b05cde-6398-4433-ba18-8ef6348b28f5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:38.860701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7065a845fd404bbfbf11c1ad766e8c09', 'request_id': 'req-58b05cde-6398-4433-ba18-8ef6348b28f5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:38.867360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7065a845fd404bbfbf11c1ad766e8c09', 'request_id': 'req-58b05cde-6398-4433-ba18-8ef6348b28f5'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:38.881128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] The token cache is being invalidate because project 7065a845fd404bbfbf11c1ad766e8c09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:38.881419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:38.882154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:38.882562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ada6a8-0635-414d-8cbb-3b2191b16366 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.883748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 928/1871] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:38 2022] DELETE /identity/v3/projects/7065a845fd404bbfbf11c1ad766e8c09 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:38.887021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:38.888755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:38.950947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3eb8f0-f5ba-476d-8192-73669bec3e25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.951860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eb8f0-f5ba-476d-8192-73669bec3e25 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.952535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eb8f0-f5ba-476d-8192-73669bec3e25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.952973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3eb8f0-f5ba-476d-8192-73669bec3e25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.954243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eb8f0-f5ba-476d-8192-73669bec3e25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.958610 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3eb8f0-f5ba-476d-8192-73669bec3e25 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.970572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b05cde-6398-4433-ba18-8ef6348b28f5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:38.977639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651116-4385-41e4-b1d3-1bfb053c6879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:38.978247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651116-4385-41e4-b1d3-1bfb053c6879 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.978544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651116-4385-41e4-b1d3-1bfb053c6879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.978837 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7651116-4385-41e4-b1d3-1bfb053c6879 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:38.979627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651116-4385-41e4-b1d3-1bfb053c6879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:38.981699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7651116-4385-41e4-b1d3-1bfb053c6879 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:38.991554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 944/1872] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:38 2022] POST /identity/v3/projects => generated 387 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:38.993185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26806758-7c5d-43a5-b71c-583d53912bb1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:38.993581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26806758-7c5d-43a5-b71c-583d53912bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:38.994061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26806758-7c5d-43a5-b71c-583d53912bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.009203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 929/1873] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:38 2022] POST /identity/v3/users => generated 294 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:39.010284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651116-4385-41e4-b1d3-1bfb053c6879 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.012454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c7651116-4385-41e4-b1d3-1bfb053c6879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:39.022974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-26806758-7c5d-43a5-b71c-583d53912bb1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:39.024704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-26806758-7c5d-43a5-b71c-583d53912bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:39.026682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7651116-4385-41e4-b1d3-1bfb053c6879 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.102152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 945/1874] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:39.103287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26806758-7c5d-43a5-b71c-583d53912bb1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.114279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26806758-7c5d-43a5-b71c-583d53912bb1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.119479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226984f4-4671-4524-b303-ab9858e55cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.120154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226984f4-4671-4524-b303-ab9858e55cec admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.120441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226984f4-4671-4524-b303-ab9858e55cec admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.120717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226984f4-4671-4524-b303-ab9858e55cec admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.121447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226984f4-4671-4524-b303-ab9858e55cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.123070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226984f4-4671-4524-b303-ab9858e55cec admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.126547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db379e-b06b-44bd-80a1-bb976b0b2caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.127319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db379e-b06b-44bd-80a1-bb976b0b2caa admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.127679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db379e-b06b-44bd-80a1-bb976b0b2caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.128280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3db379e-b06b-44bd-80a1-bb976b0b2caa admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.129154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db379e-b06b-44bd-80a1-bb976b0b2caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.131209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3db379e-b06b-44bd-80a1-bb976b0b2caa admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.136971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 946/1875] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:39.136971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69783d-4ad7-49c7-8aad-c0978116e878 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.136971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69783d-4ad7-49c7-8aad-c0978116e878 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.136971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f69783d-4ad7-49c7-8aad-c0978116e878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.140205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:39.154374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9f69783d-4ad7-49c7-8aad-c0978116e878 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:39.155609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9f69783d-4ad7-49c7-8aad-c0978116e878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:39.161512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 930/1876] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:39 2022] POST /identity/v3/users => generated 300 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:39.161512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db379e-b06b-44bd-80a1-bb976b0b2caa admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.167439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 947/1877] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:39.168052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69783d-4ad7-49c7-8aad-c0978116e878 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.172600 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3db379e-b06b-44bd-80a1-bb976b0b2caa admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.177736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea23f1-cf99-468f-bf55-cb644a32531a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.178586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f69783d-4ad7-49c7-8aad-c0978116e878 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.179159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea23f1-cf99-468f-bf55-cb644a32531a admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.179485 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea23f1-cf99-468f-bf55-cb644a32531a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.179848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea23f1-cf99-468f-bf55-cb644a32531a admin admin] PATH_INFO: `/v3/projects/bfbe57febb0344348af2dd5d6283fb75/users/026587500988433a80af47f20940e009/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.184778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc35c9b-f5ab-48b2-b2db-a037963ea212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.185564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc35c9b-f5ab-48b2-b2db-a037963ea212 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.185851 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc35c9b-f5ab-48b2-b2db-a037963ea212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.186158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc35c9b-f5ab-48b2-b2db-a037963ea212 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.186898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc35c9b-f5ab-48b2-b2db-a037963ea212 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.188536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc35c9b-f5ab-48b2-b2db-a037963ea212 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.197803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea23f1-cf99-468f-bf55-cb644a32531a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbe57febb0344348af2dd5d6283fb75, user_id=026587500988433a80af47f20940e009, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.203101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 948/1878] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:39.203236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbea23f1-cf99-468f-bf55-cb644a32531a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.203326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc35c9b-f5ab-48b2-b2db-a037963ea212 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.208001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:39.214589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 931/1879] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:39 2022] PUT /identity/v3/projects/bfbe57febb0344348af2dd5d6283fb75/users/026587500988433a80af47f20940e009/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:39.214818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc35c9b-f5ab-48b2-b2db-a037963ea212 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.215388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea23f1-cf99-468f-bf55-cb644a32531a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.279458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea23f1-cf99-468f-bf55-cb644a32531a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.284626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26f4ec-ea6b-4fd0-b2d4-df0228db779b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.285517 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26f4ec-ea6b-4fd0-b2d4-df0228db779b admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.285791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26f4ec-ea6b-4fd0-b2d4-df0228db779b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.286090 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26f4ec-ea6b-4fd0-b2d4-df0228db779b admin admin] PATH_INFO: `/v3/projects/56270106277240188969dcf435f8e955/users/cfbf1d609f1942c9bee5e781b30fccba/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.288692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109923e-a52d-4821-aa16-af6ccaaec34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.289438 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109923e-a52d-4821-aa16-af6ccaaec34b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.289798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109923e-a52d-4821-aa16-af6ccaaec34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.290175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109923e-a52d-4821-aa16-af6ccaaec34b admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.291026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109923e-a52d-4821-aa16-af6ccaaec34b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:39.291407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109923e-a52d-4821-aa16-af6ccaaec34b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.293133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b109923e-a52d-4821-aa16-af6ccaaec34b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.300820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26f4ec-ea6b-4fd0-b2d4-df0228db779b admin admin] RBAC: Authorizing `identity:create_grant(project_id=56270106277240188969dcf435f8e955, user_id=cfbf1d609f1942c9bee5e781b30fccba, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.303653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26f4ec-ea6b-4fd0-b2d4-df0228db779b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.305832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 949/1880] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:05:39.306693 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109923e-a52d-4821-aa16-af6ccaaec34b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.316549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109923e-a52d-4821-aa16-af6ccaaec34b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.319445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 932/1881] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:39 2022] PUT /identity/v3/projects/56270106277240188969dcf435f8e955/users/cfbf1d609f1942c9bee5e781b30fccba/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:39.320368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26f4ec-ea6b-4fd0-b2d4-df0228db779b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.376398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25e0f-2c9c-4a65-90bc-2c56d75a4246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.377074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25e0f-2c9c-4a65-90bc-2c56d75a4246 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.377363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25e0f-2c9c-4a65-90bc-2c56d75a4246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.377643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b25e0f-2c9c-4a65-90bc-2c56d75a4246 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.378653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b25e0f-2c9c-4a65-90bc-2c56d75a4246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.380273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b25e0f-2c9c-4a65-90bc-2c56d75a4246 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.383135 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26f4ec-ea6b-4fd0-b2d4-df0228db779b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.387694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592f3c3-3b59-4986-87b5-debb64ccd4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.388352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592f3c3-3b59-4986-87b5-debb64ccd4ee admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.388635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592f3c3-3b59-4986-87b5-debb64ccd4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.388931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592f3c3-3b59-4986-87b5-debb64ccd4ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.389621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592f3c3-3b59-4986-87b5-debb64ccd4ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.391232 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 950/1882] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:39.392151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25e0f-2c9c-4a65-90bc-2c56d75a4246 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.392256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592f3c3-3b59-4986-87b5-debb64ccd4ee admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.401677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b25e0f-2c9c-4a65-90bc-2c56d75a4246 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.407797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4e4d0-d9bf-4ee8-9738-8c8291ed66c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.408574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4e4d0-d9bf-4ee8-9738-8c8291ed66c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.408909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4e4d0-d9bf-4ee8-9738-8c8291ed66c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.409182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c4e4d0-d9bf-4ee8-9738-8c8291ed66c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.409939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4e4d0-d9bf-4ee8-9738-8c8291ed66c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.411609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c4e4d0-d9bf-4ee8-9738-8c8291ed66c0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.418645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 933/1883] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:39 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:39.419290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592f3c3-3b59-4986-87b5-debb64ccd4ee admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.425129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 951/1884] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:39.426570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4e4d0-d9bf-4ee8-9738-8c8291ed66c0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.436714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c4e4d0-d9bf-4ee8-9738-8c8291ed66c0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.472892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:39.503745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592f3c3-3b59-4986-87b5-debb64ccd4ee admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.510327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea42ab8-621e-4e41-83fd-0dfc2d0245ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.511344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea42ab8-621e-4e41-83fd-0dfc2d0245ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.511699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea42ab8-621e-4e41-83fd-0dfc2d0245ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.512046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea42ab8-621e-4e41-83fd-0dfc2d0245ba admin admin] PATH_INFO: `/v3/projects/bfbe57febb0344348af2dd5d6283fb75/users/026587500988433a80af47f20940e009/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.514167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea42ab8-621e-4e41-83fd-0dfc2d0245ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbe57febb0344348af2dd5d6283fb75, user_id=026587500988433a80af47f20940e009, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.515992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ab9ee8-b341-40a5-a80b-6d032464f801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.516724 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab9ee8-b341-40a5-a80b-6d032464f801 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.517014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab9ee8-b341-40a5-a80b-6d032464f801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.517294 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ab9ee8-b341-40a5-a80b-6d032464f801 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.517831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea42ab8-621e-4e41-83fd-0dfc2d0245ba admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.518136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab9ee8-b341-40a5-a80b-6d032464f801 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.520391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ab9ee8-b341-40a5-a80b-6d032464f801 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.538085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 934/1885] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:39 2022] PUT /identity/v3/projects/bfbe57febb0344348af2dd5d6283fb75/users/026587500988433a80af47f20940e009/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:39.538085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea42ab8-621e-4e41-83fd-0dfc2d0245ba admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.554904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 952/1886] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:39 2022] POST /identity/v3/users => generated 283 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:39.558153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976c684-3035-46ae-8ee4-417bedb967e5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.560043 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976c684-3035-46ae-8ee4-417bedb967e5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.560332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976c684-3035-46ae-8ee4-417bedb967e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.582814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:39.592184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9976c684-3035-46ae-8ee4-417bedb967e5 None None] MFA Rules not processed for user `026587500988433a80af47f20940e009`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:39.595147 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9976c684-3035-46ae-8ee4-417bedb967e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:39.619150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea42ab8-621e-4e41-83fd-0dfc2d0245ba admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.625804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595e33c-7963-46f4-9ab0-acf2291c6a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.627028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595e33c-7963-46f4-9ab0-acf2291c6a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.627381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595e33c-7963-46f4-9ab0-acf2291c6a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.627722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595e33c-7963-46f4-9ab0-acf2291c6a09 admin admin] PATH_INFO: `/v3/projects/56270106277240188969dcf435f8e955/users/cfbf1d609f1942c9bee5e781b30fccba/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.630279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595e33c-7963-46f4-9ab0-acf2291c6a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56270106277240188969dcf435f8e955, user_id=cfbf1d609f1942c9bee5e781b30fccba, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.634427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c595e33c-7963-46f4-9ab0-acf2291c6a09 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.654621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 935/1887] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:39 2022] PUT /identity/v3/projects/56270106277240188969dcf435f8e955/users/cfbf1d609f1942c9bee5e781b30fccba/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:39.657723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595e33c-7963-46f4-9ab0-acf2291c6a09 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.692666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 953/1888] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:39 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:39.694085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505b37b-cea1-4aea-b722-f05f9438a63a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.694378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505b37b-cea1-4aea-b722-f05f9438a63a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.694650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505b37b-cea1-4aea-b722-f05f9438a63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.722681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c505b37b-cea1-4aea-b722-f05f9438a63a None None] MFA Rules not processed for user `cfbf1d609f1942c9bee5e781b30fccba`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:39.723888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c505b37b-cea1-4aea-b722-f05f9438a63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:39.731589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595e33c-7963-46f4-9ab0-acf2291c6a09 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.736375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7919bcb-bae2-4dff-8fa0-67856f4682bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.737106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919bcb-bae2-4dff-8fa0-67856f4682bf admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.737456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919bcb-bae2-4dff-8fa0-67856f4682bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.737980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7919bcb-bae2-4dff-8fa0-67856f4682bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.738801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7919bcb-bae2-4dff-8fa0-67856f4682bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.740390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7919bcb-bae2-4dff-8fa0-67856f4682bf admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.751752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 936/1889] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:39.753467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7919bcb-bae2-4dff-8fa0-67856f4682bf admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.755421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d7919bcb-bae2-4dff-8fa0-67856f4682bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:39.808515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 954/1890] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:39 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:39.809453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505b37b-cea1-4aea-b722-f05f9438a63a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.822031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7919bcb-bae2-4dff-8fa0-67856f4682bf admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.823555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505b37b-cea1-4aea-b722-f05f9438a63a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.827805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698c77c-1865-42a2-a73e-d763cdc596d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.828947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698c77c-1865-42a2-a73e-d763cdc596d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.829234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698c77c-1865-42a2-a73e-d763cdc596d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.829342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d423d-9d84-4fe2-90e7-1ed526e2d190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.829504 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698c77c-1865-42a2-a73e-d763cdc596d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.829954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d423d-9d84-4fe2-90e7-1ed526e2d190 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.830365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d423d-9d84-4fe2-90e7-1ed526e2d190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.830704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d423d-9d84-4fe2-90e7-1ed526e2d190 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.831443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c698c77c-1865-42a2-a73e-d763cdc596d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:39.831577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d423d-9d84-4fe2-90e7-1ed526e2d190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.833263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d423d-9d84-4fe2-90e7-1ed526e2d190 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.847640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 955/1891] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:39.847640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d423d-9d84-4fe2-90e7-1ed526e2d190 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.854911 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c48d423d-9d84-4fe2-90e7-1ed526e2d190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:39.859089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:39.864778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d423d-9d84-4fe2-90e7-1ed526e2d190 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.868803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1c6a4-3cb3-4e27-bd20-4266bba2afd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.869495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1c6a4-3cb3-4e27-bd20-4266bba2afd5 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.869748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1c6a4-3cb3-4e27-bd20-4266bba2afd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.870921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1c6a4-3cb3-4e27-bd20-4266bba2afd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.883137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1c6a4-3cb3-4e27-bd20-4266bba2afd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.884658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1c6a4-3cb3-4e27-bd20-4266bba2afd5 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.896702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 956/1892] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:39.897471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1c6a4-3cb3-4e27-bd20-4266bba2afd5 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:39.913739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1c6a4-3cb3-4e27-bd20-4266bba2afd5 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:39.915857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698c77c-1865-42a2-a73e-d763cdc596d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.917757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698c77c-1865-42a2-a73e-d763cdc596d7 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.926727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef395d51-3d55-4167-86c1-f18d07e32682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:39.928427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef395d51-3d55-4167-86c1-f18d07e32682 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.928883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef395d51-3d55-4167-86c1-f18d07e32682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.929166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 937/1893] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:39.929315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef395d51-3d55-4167-86c1-f18d07e32682 admin admin] PATH_INFO: `/v3/projects/0c5da2134eef4275868694db9e57bba7/users/cb9951f1b8b54b369f170df42de28220/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:39.951242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef395d51-3d55-4167-86c1-f18d07e32682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c5da2134eef4275868694db9e57bba7, user_id=cb9951f1b8b54b369f170df42de28220, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:39.956913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef395d51-3d55-4167-86c1-f18d07e32682 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:39.991458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 957/1894] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:39 2022] PUT /identity/v3/projects/0c5da2134eef4275868694db9e57bba7/users/cb9951f1b8b54b369f170df42de28220/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:39.993531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbaa523-d43c-4a55-8666-c13943634604 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:39.993737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbaa523-d43c-4a55-8666-c13943634604 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:39.993975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbaa523-d43c-4a55-8666-c13943634604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.031015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2fbaa523-d43c-4a55-8666-c13943634604 None None] MFA Rules not processed for user `cb9951f1b8b54b369f170df42de28220`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:40.032244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2fbaa523-d43c-4a55-8666-c13943634604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:40.078504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef395d51-3d55-4167-86c1-f18d07e32682 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:40.122555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 938/1895] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:39 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:40.127183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbaa523-d43c-4a55-8666-c13943634604 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:40.141778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:40.179666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef395d51-3d55-4167-86c1-f18d07e32682 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:40.185395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:40.186668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.187894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.188329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] PATH_INFO: `/v3/users/7fd97262fc464caab3e1bc6252c77f25` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.189903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd97262fc464caab3e1bc6252c77f25)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:40.192186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:40.222325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbaa523-d43c-4a55-8666-c13943634604 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:40.228241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204804c4-78cb-4f57-8368-08993f313445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:40.228894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204804c4-78cb-4f57-8368-08993f313445 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.229192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204804c4-78cb-4f57-8368-08993f313445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.229460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204804c4-78cb-4f57-8368-08993f313445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.230228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204804c4-78cb-4f57-8368-08993f313445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:40.232596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204804c4-78cb-4f57-8368-08993f313445 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:40.254592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd97262fc464caab3e1bc6252c77f25'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:40.256140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd97262fc464caab3e1bc6252c77f25'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:40.264752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd97262fc464caab3e1bc6252c77f25'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:40.267422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 939/1896] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:40 2022] POST /identity/v3/projects => generated 365 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:40.278441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204804c4-78cb-4f57-8368-08993f313445 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:40.282032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd97262fc464caab3e1bc6252c77f25'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:40.282735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2fad1a75-07ef-4efc-9a38-2591f75bc0bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd97262fc464caab3e1bc6252c77f25'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:40.322252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 958/1897] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:40 2022] DELETE /identity/v3/users/7fd97262fc464caab3e1bc6252c77f25 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:40.326816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dfce9-0364-4c62-9e6a-46408914c29d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.327104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dfce9-0364-4c62-9e6a-46408914c29d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.327405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8dfce9-0364-4c62-9e6a-46408914c29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.365008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ee8dfce9-0364-4c62-9e6a-46408914c29d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:40.366212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ee8dfce9-0364-4c62-9e6a-46408914c29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:40.367673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:40.424945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204804c4-78cb-4f57-8368-08993f313445 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:40.446151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 959/1898] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:40.457321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dfce9-0364-4c62-9e6a-46408914c29d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:40.458868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ee8dfce9-0364-4c62-9e6a-46408914c29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:40.488161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984b534-8107-4c31-9dea-e91f3fb61cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:40.489252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984b534-8107-4c31-9dea-e91f3fb61cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.489542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984b534-8107-4c31-9dea-e91f3fb61cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.489821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c984b534-8107-4c31-9dea-e91f3fb61cbf admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.490928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984b534-8107-4c31-9dea-e91f3fb61cbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:40.493332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c984b534-8107-4c31-9dea-e91f3fb61cbf admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:40.521777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 940/1899] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:40 2022] POST /identity/v3/users => generated 289 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:40.525817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984b534-8107-4c31-9dea-e91f3fb61cbf admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:40.525999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8dfce9-0364-4c62-9e6a-46408914c29d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:40.527723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c984b534-8107-4c31-9dea-e91f3fb61cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:40.530384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78be6b1-d665-4c52-8e3c-df65fc482ab3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:40.531566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78be6b1-d665-4c52-8e3c-df65fc482ab3 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.531850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78be6b1-d665-4c52-8e3c-df65fc482ab3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.532128 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78be6b1-d665-4c52-8e3c-df65fc482ab3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.540075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:40.542984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c984b534-8107-4c31-9dea-e91f3fb61cbf admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:40.543622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78be6b1-d665-4c52-8e3c-df65fc482ab3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:40.548321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78be6b1-d665-4c52-8e3c-df65fc482ab3 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:40.549443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084ea1a-9d12-4732-9558-d7160ac2e728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:40.550162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084ea1a-9d12-4732-9558-d7160ac2e728 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.550446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084ea1a-9d12-4732-9558-d7160ac2e728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.550720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7084ea1a-9d12-4732-9558-d7160ac2e728 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.551455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084ea1a-9d12-4732-9558-d7160ac2e728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:40.553039 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7084ea1a-9d12-4732-9558-d7160ac2e728 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:40.570081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 941/1900] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:40.571404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 960/1901] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:40.572770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084ea1a-9d12-4732-9558-d7160ac2e728 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:40.582550 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7084ea1a-9d12-4732-9558-d7160ac2e728 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:40.587299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e698c-6d82-4b06-b950-f8eac8c43ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:40.588215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e698c-6d82-4b06-b950-f8eac8c43ab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.588499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e698c-6d82-4b06-b950-f8eac8c43ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.588776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e698c-6d82-4b06-b950-f8eac8c43ab4 admin admin] PATH_INFO: `/v3/projects/4648be0848a541e492addcfbbcbeee01/users/b67db750e2b1475c9ea724ce2dacebbb/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.604381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e698c-6d82-4b06-b950-f8eac8c43ab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4648be0848a541e492addcfbbcbeee01, user_id=b67db750e2b1475c9ea724ce2dacebbb, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:40.607069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e698c-6d82-4b06-b950-f8eac8c43ab4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:40.622175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 942/1902] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:40 2022] PUT /identity/v3/projects/4648be0848a541e492addcfbbcbeee01/users/b67db750e2b1475c9ea724ce2dacebbb/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:40.626951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78be6b1-d665-4c52-8e3c-df65fc482ab3 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:40.637344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78be6b1-d665-4c52-8e3c-df65fc482ab3 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:40.700189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a7acb7-7a77-4de9-996a-2ed2769a2699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:40.700878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7acb7-7a77-4de9-996a-2ed2769a2699 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.701238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7acb7-7a77-4de9-996a-2ed2769a2699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.701421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a7acb7-7a77-4de9-996a-2ed2769a2699 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.702223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7acb7-7a77-4de9-996a-2ed2769a2699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:40.703843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a7acb7-7a77-4de9-996a-2ed2769a2699 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:40.718959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 961/1903] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:40.724243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e698c-6d82-4b06-b950-f8eac8c43ab4 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:40.738832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e698c-6d82-4b06-b950-f8eac8c43ab4 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:40.745770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050355e4-78e3-421c-abc8-f1a267f9659c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:40.747080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050355e4-78e3-421c-abc8-f1a267f9659c admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.747200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050355e4-78e3-421c-abc8-f1a267f9659c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.747566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050355e4-78e3-421c-abc8-f1a267f9659c admin admin] PATH_INFO: `/v3/projects/4648be0848a541e492addcfbbcbeee01/users/b67db750e2b1475c9ea724ce2dacebbb/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.750337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050355e4-78e3-421c-abc8-f1a267f9659c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4648be0848a541e492addcfbbcbeee01, user_id=b67db750e2b1475c9ea724ce2dacebbb, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:40.756196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050355e4-78e3-421c-abc8-f1a267f9659c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:40.770869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 943/1904] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:40 2022] PUT /identity/v3/projects/4648be0848a541e492addcfbbcbeee01/users/b67db750e2b1475c9ea724ce2dacebbb/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:40.781354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6cb33-a734-4d87-994a-fab014ccdd80 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.781672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6cb33-a734-4d87-994a-fab014ccdd80 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.781954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6cb33-a734-4d87-994a-fab014ccdd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.811046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-26d6cb33-a734-4d87-994a-fab014ccdd80 None None] MFA Rules not processed for user `b67db750e2b1475c9ea724ce2dacebbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:40.812450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-26d6cb33-a734-4d87-994a-fab014ccdd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:40.890715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 962/1905] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:40 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:40.902309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050355e4-78e3-421c-abc8-f1a267f9659c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:40.904219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-050355e4-78e3-421c-abc8-f1a267f9659c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:40.962437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050355e4-78e3-421c-abc8-f1a267f9659c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:40.966096 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882a4f-605e-4dad-bd82-889b04c37640 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:40.966750 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882a4f-605e-4dad-bd82-889b04c37640 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:40.967040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882a4f-605e-4dad-bd82-889b04c37640 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:40.967316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da882a4f-605e-4dad-bd82-889b04c37640 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:40.977347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da882a4f-605e-4dad-bd82-889b04c37640 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:40.979009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da882a4f-605e-4dad-bd82-889b04c37640 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:40.989529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 944/1906] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:40 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:41.117669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6cb33-a734-4d87-994a-fab014ccdd80 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:41.119593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-26d6cb33-a734-4d87-994a-fab014ccdd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:41.203198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6cb33-a734-4d87-994a-fab014ccdd80 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:41.211453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:41.212398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:41.212754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:41.213148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] PATH_INFO: `/v3/projects/61aeeea6c3ab471f8c227b80a4be7fb0` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:41.214451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61aeeea6c3ab471f8c227b80a4be7fb0)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:41.216332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:41.245235 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:41.278794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61aeeea6c3ab471f8c227b80a4be7fb0', 'request_id': 'req-dc1d8374-f465-49b0-aeb2-3e07c8f43150'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.285234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61aeeea6c3ab471f8c227b80a4be7fb0', 'request_id': 'req-dc1d8374-f465-49b0-aeb2-3e07c8f43150'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.293552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61aeeea6c3ab471f8c227b80a4be7fb0', 'request_id': 'req-dc1d8374-f465-49b0-aeb2-3e07c8f43150'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.306363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] The token cache is being invalidate because project 61aeeea6c3ab471f8c227b80a4be7fb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:41.306657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.307318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.308871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 963/1907] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:41 2022] DELETE /identity/v3/projects/61aeeea6c3ab471f8c227b80a4be7fb0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:41.313364 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882a4f-605e-4dad-bd82-889b04c37640 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:41.323752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da882a4f-605e-4dad-bd82-889b04c37640 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:41.340326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:41.390794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:41.393105 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:41.393446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:41.393744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] PATH_INFO: `/v3/users/cfbf1d609f1942c9bee5e781b30fccba` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:41.395636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfbf1d609f1942c9bee5e781b30fccba)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:41.397796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:41.488058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfbf1d609f1942c9bee5e781b30fccba'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.489043 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfbf1d609f1942c9bee5e781b30fccba'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.489882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfbf1d609f1942c9bee5e781b30fccba'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.506976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:41.512987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfbf1d609f1942c9bee5e781b30fccba'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.513635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:41.523664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfbf1d609f1942c9bee5e781b30fccba'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.530426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1d8374-f465-49b0-aeb2-3e07c8f43150 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:41.539345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 945/1908] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:41 2022] DELETE /identity/v3/users/cfbf1d609f1942c9bee5e781b30fccba => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:41.574780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:41.575682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:41.575962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:41.576234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] PATH_INFO: `/v3/users/cb9951f1b8b54b369f170df42de28220` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:41.577515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb9951f1b8b54b369f170df42de28220)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:41.580032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:41.616634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:41.618366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:41.680200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb9951f1b8b54b369f170df42de28220'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.680873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb9951f1b8b54b369f170df42de28220'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.689475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb9951f1b8b54b369f170df42de28220'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.699808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb9951f1b8b54b369f170df42de28220'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.700709 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4247eb01-c8dc-4d7e-ad05-3cff8bb8f6be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb9951f1b8b54b369f170df42de28220'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:41.727146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 964/1909] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:41 2022] DELETE /identity/v3/users/cb9951f1b8b54b369f170df42de28220 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:41.730073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d803fe8-e2d7-4575-af30-9f5242f691d2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:41.731962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd748de4-5976-4fb2-b2bf-aab420ea824f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:41.732859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd748de4-5976-4fb2-b2bf-aab420ea824f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:41.733291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd748de4-5976-4fb2-b2bf-aab420ea824f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:41.771258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cd748de4-5976-4fb2-b2bf-aab420ea824f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:41.772498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cd748de4-5976-4fb2-b2bf-aab420ea824f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:41.809102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7960d-39f6-4364-b877-51848001a5db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:41.809810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7960d-39f6-4364-b877-51848001a5db service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:41.810124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7960d-39f6-4364-b877-51848001a5db service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:41.810407 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7960d-39f6-4364-b877-51848001a5db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:41.812904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-24b7960d-39f6-4364-b877-51848001a5db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:41.826502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:41.866353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 965/1910] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:41.881460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd748de4-5976-4fb2-b2bf-aab420ea824f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:41.883563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cd748de4-5976-4fb2-b2bf-aab420ea824f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:41.902567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7960d-39f6-4364-b877-51848001a5db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:41.904989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7960d-39f6-4364-b877-51848001a5db service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:41.912710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:41.935326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 946/1911] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:41.979300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd748de4-5976-4fb2-b2bf-aab420ea824f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:41.985803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8fde4-775b-4f47-b603-14f6e3110b38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:41.993484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8fde4-775b-4f47-b603-14f6e3110b38 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:41.993962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8fde4-775b-4f47-b603-14f6e3110b38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:41.994380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8fde4-775b-4f47-b603-14f6e3110b38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.006844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8fde4-775b-4f47-b603-14f6e3110b38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:42.008739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8fde4-775b-4f47-b603-14f6e3110b38 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:42.022200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 966/1912] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:42.067153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7960d-39f6-4364-b877-51848001a5db service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:42.071241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-24b7960d-39f6-4364-b877-51848001a5db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:42.131991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7960d-39f6-4364-b877-51848001a5db service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:42.148380 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:42.149535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:42.150001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:42.150500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] PATH_INFO: `/v3/projects/56270106277240188969dcf435f8e955` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.151897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56270106277240188969dcf435f8e955)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:42.154382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:42.240524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56270106277240188969dcf435f8e955', 'request_id': 'req-7e76773a-4ad0-4661-857e-5d751c225565'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.252263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56270106277240188969dcf435f8e955', 'request_id': 'req-7e76773a-4ad0-4661-857e-5d751c225565'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.260696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56270106277240188969dcf435f8e955', 'request_id': 'req-7e76773a-4ad0-4661-857e-5d751c225565'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.285700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] The token cache is being invalidate because project 56270106277240188969dcf435f8e955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:42.286319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.287260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.289029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 947/1913] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:42 2022] DELETE /identity/v3/projects/56270106277240188969dcf435f8e955 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:42.295256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e32422-20d6-4823-b16e-11a7a401dd32 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:42.295687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e32422-20d6-4823-b16e-11a7a401dd32 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:42.295968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e32422-20d6-4823-b16e-11a7a401dd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.324770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-74e32422-20d6-4823-b16e-11a7a401dd32 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:42.326681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-74e32422-20d6-4823-b16e-11a7a401dd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:42.422131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 967/1914] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:42.422499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:42.423250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25d85d-0023-4294-bf16-b5bf7d17a7b9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:42.423616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25d85d-0023-4294-bf16-b5bf7d17a7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:42.423731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:42.423987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c25d85d-0023-4294-bf16-b5bf7d17a7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.452104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76773a-4ad0-4661-857e-5d751c225565 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:42.461134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:42.463029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:42.463402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:42.463789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] PATH_INFO: `/v3/projects/0c5da2134eef4275868694db9e57bba7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.466828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c5da2134eef4275868694db9e57bba7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:42.470794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:42.476520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6c25d85d-0023-4294-bf16-b5bf7d17a7b9 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:42.477885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6c25d85d-0023-4294-bf16-b5bf7d17a7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:42.487566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 968/1915] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:42.492299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25d85d-0023-4294-bf16-b5bf7d17a7b9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:42.504630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c25d85d-0023-4294-bf16-b5bf7d17a7b9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:42.510702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51874c-fae3-4b5b-92c7-d526bfa3eb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:42.511886 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51874c-fae3-4b5b-92c7-d526bfa3eb9b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:42.512243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51874c-fae3-4b5b-92c7-d526bfa3eb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:42.512473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51874c-fae3-4b5b-92c7-d526bfa3eb9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.513230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51874c-fae3-4b5b-92c7-d526bfa3eb9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:42.513654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51874c-fae3-4b5b-92c7-d526bfa3eb9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:42.515984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51874c-fae3-4b5b-92c7-d526bfa3eb9b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:42.528711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 969/1916] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:05:42.533484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51874c-fae3-4b5b-92c7-d526bfa3eb9b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:42.534255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c5da2134eef4275868694db9e57bba7', 'request_id': 'req-2085da93-5d26-423d-9904-f9b4e3ec2902'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.542386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c5da2134eef4275868694db9e57bba7', 'request_id': 'req-2085da93-5d26-423d-9904-f9b4e3ec2902'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.548532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c5da2134eef4275868694db9e57bba7', 'request_id': 'req-2085da93-5d26-423d-9904-f9b4e3ec2902'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.550314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51874c-fae3-4b5b-92c7-d526bfa3eb9b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:42.563042 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] The token cache is being invalidate because project 0c5da2134eef4275868694db9e57bba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:42.563408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.564163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.566112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 948/1917] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:42 2022] DELETE /identity/v3/projects/0c5da2134eef4275868694db9e57bba7 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:42.570332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:42.572957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:42.598782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:42.614640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8c77a-89ca-47ea-a3f9-326c88b9ec76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:42.615316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8c77a-89ca-47ea-a3f9-326c88b9ec76 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:42.615602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8c77a-89ca-47ea-a3f9-326c88b9ec76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:42.615882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c8c77a-89ca-47ea-a3f9-326c88b9ec76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.616654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8c77a-89ca-47ea-a3f9-326c88b9ec76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:42.618570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c8c77a-89ca-47ea-a3f9-326c88b9ec76 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:42.648720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085da93-5d26-423d-9904-f9b4e3ec2902 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:42.653456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:42.654376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:42.654657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:42.655019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] PATH_INFO: `/v3/users/b67db750e2b1475c9ea724ce2dacebbb` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.655406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 970/1918] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:42 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:42.659760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8c77a-89ca-47ea-a3f9-326c88b9ec76 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:42.660019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b67db750e2b1475c9ea724ce2dacebbb)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:42.662425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f4c8c77a-89ca-47ea-a3f9-326c88b9ec76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:42.662618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:42.701801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:42.757821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c8c77a-89ca-47ea-a3f9-326c88b9ec76 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:42.768824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b67db750e2b1475c9ea724ce2dacebbb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.770069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b67db750e2b1475c9ea724ce2dacebbb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.770920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1544c6-95ee-469a-b63a-7813f8d3c8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:42.771123 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b67db750e2b1475c9ea724ce2dacebbb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.771625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1544c6-95ee-469a-b63a-7813f8d3c8da admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:42.771914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1544c6-95ee-469a-b63a-7813f8d3c8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:42.772196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1544c6-95ee-469a-b63a-7813f8d3c8da admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.774505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1544c6-95ee-469a-b63a-7813f8d3c8da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:42.779362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1544c6-95ee-469a-b63a-7813f8d3c8da admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:42.786464 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b67db750e2b1475c9ea724ce2dacebbb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.801205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b67db750e2b1475c9ea724ce2dacebbb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:42.824854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 949/1919] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:42 2022] DELETE /identity/v3/users/b67db750e2b1475c9ea724ce2dacebbb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:42.825744 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 971/1920] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:42 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:42.826259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:42.827651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:42.833381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1544c6-95ee-469a-b63a-7813f8d3c8da admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:42.841025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fa1544c6-95ee-469a-b63a-7813f8d3c8da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:42.867723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1544c6-95ee-469a-b63a-7813f8d3c8da admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:42.921084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d64de-ff00-4fb7-a181-a26f7b7aaa7a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:42.925686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd394be-c8eb-46c5-bfa6-48eb07f290e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:42.926630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd394be-c8eb-46c5-bfa6-48eb07f290e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:42.927246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd394be-c8eb-46c5-bfa6-48eb07f290e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:42.927525 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd394be-c8eb-46c5-bfa6-48eb07f290e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.929496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-afd394be-c8eb-46c5-bfa6-48eb07f290e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:42.990525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61475a-dd0d-46a4-9f68-0d3b698c157a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:42.991179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:42.991291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61475a-dd0d-46a4-9f68-0d3b698c157a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:42.991486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61475a-dd0d-46a4-9f68-0d3b698c157a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:42.991758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61475a-dd0d-46a4-9f68-0d3b698c157a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:42.992486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61475a-dd0d-46a4-9f68-0d3b698c157a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:42.995670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61475a-dd0d-46a4-9f68-0d3b698c157a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.007184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 972/1921] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:42 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:43.011248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61475a-dd0d-46a4-9f68-0d3b698c157a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.015419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd394be-c8eb-46c5-bfa6-48eb07f290e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.016927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd394be-c8eb-46c5-bfa6-48eb07f290e5 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.028256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 950/1922] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:42 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:43.035053 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61475a-dd0d-46a4-9f68-0d3b698c157a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.041267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b917b-eb7c-49ff-9d39-1f76a80d7181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.042032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b917b-eb7c-49ff-9d39-1f76a80d7181 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.042415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b917b-eb7c-49ff-9d39-1f76a80d7181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.042691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b917b-eb7c-49ff-9d39-1f76a80d7181 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.043933 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b917b-eb7c-49ff-9d39-1f76a80d7181 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.046007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b917b-eb7c-49ff-9d39-1f76a80d7181 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.066120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 973/1923] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:43.068248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd394be-c8eb-46c5-bfa6-48eb07f290e5 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.082272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd394be-c8eb-46c5-bfa6-48eb07f290e5 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.091063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f58bc-ce0b-4281-9c57-7bebe0965bfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.092254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f58bc-ce0b-4281-9c57-7bebe0965bfc admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.092766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f58bc-ce0b-4281-9c57-7bebe0965bfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.092989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f58bc-ce0b-4281-9c57-7bebe0965bfc admin admin] PATH_INFO: `/v3/projects/ab1e05f63e5d449f91ace0121205c1ea/users/132f18d29706457dadc37ccbc9f2dc75/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.131309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f58bc-ce0b-4281-9c57-7bebe0965bfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab1e05f63e5d449f91ace0121205c1ea, user_id=132f18d29706457dadc37ccbc9f2dc75, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.133991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f58bc-ce0b-4281-9c57-7bebe0965bfc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.159002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 951/1924] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:43 2022] PUT /identity/v3/projects/ab1e05f63e5d449f91ace0121205c1ea/users/132f18d29706457dadc37ccbc9f2dc75/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:43.170356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5f83a-fcf8-4413-968d-49756473d1ab None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.170546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5f83a-fcf8-4413-968d-49756473d1ab None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.170835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f5f83a-fcf8-4413-968d-49756473d1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.213180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2f5f83a-fcf8-4413-968d-49756473d1ab None None] MFA Rules not processed for user `132f18d29706457dadc37ccbc9f2dc75`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:43.214393 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2f5f83a-fcf8-4413-968d-49756473d1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:43.270691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f58bc-ce0b-4281-9c57-7bebe0965bfc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.272551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-010f58bc-ce0b-4281-9c57-7bebe0965bfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:43.302195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 974/1925] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:43.303102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5f83a-fcf8-4413-968d-49756473d1ab None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.341221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:43.344380 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f58bc-ce0b-4281-9c57-7bebe0965bfc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.350113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.351460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.351460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.351634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] PATH_INFO: `/v3/projects/4648be0848a541e492addcfbbcbeee01` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.352836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4648be0848a541e492addcfbbcbeee01)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.354791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.372272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f5f83a-fcf8-4413-968d-49756473d1ab None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.376892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a3ef7-7492-4a18-9140-10e899c1904e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.377616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a3ef7-7492-4a18-9140-10e899c1904e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.377885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a3ef7-7492-4a18-9140-10e899c1904e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.378449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a3ef7-7492-4a18-9140-10e899c1904e admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.379190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a3ef7-7492-4a18-9140-10e899c1904e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.381167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a3ef7-7492-4a18-9140-10e899c1904e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.408047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4648be0848a541e492addcfbbcbeee01', 'request_id': 'req-9f26e57d-c9a9-498d-8914-1ec723b6d322'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:43.408686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 975/1926] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:43.412922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a3ef7-7492-4a18-9140-10e899c1904e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.415511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4648be0848a541e492addcfbbcbeee01', 'request_id': 'req-9f26e57d-c9a9-498d-8914-1ec723b6d322'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:43.423886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4648be0848a541e492addcfbbcbeee01', 'request_id': 'req-9f26e57d-c9a9-498d-8914-1ec723b6d322'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:43.435633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] The token cache is being invalidate because project 4648be0848a541e492addcfbbcbeee01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:43.435979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:43.436666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9f26e57d-c9a9-498d-8914-1ec723b6d322 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:43.438661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 952/1927] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:43 2022] DELETE /identity/v3/projects/4648be0848a541e492addcfbbcbeee01 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:43.443729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba513c7-89f1-48bc-a674-fd10df51df71 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.444025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba513c7-89f1-48bc-a674-fd10df51df71 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.444492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba513c7-89f1-48bc-a674-fd10df51df71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.465594 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2ba513c7-89f1-48bc-a674-fd10df51df71 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:43.468105 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2ba513c7-89f1-48bc-a674-fd10df51df71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:43.486923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a3ef7-7492-4a18-9140-10e899c1904e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.492604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463979c-1d3e-41dd-aefc-d11f9bc8f181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.493895 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463979c-1d3e-41dd-aefc-d11f9bc8f181 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.494206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463979c-1d3e-41dd-aefc-d11f9bc8f181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.494484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1463979c-1d3e-41dd-aefc-d11f9bc8f181 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.495167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463979c-1d3e-41dd-aefc-d11f9bc8f181 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.497874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1463979c-1d3e-41dd-aefc-d11f9bc8f181 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.518305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:43.528804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 976/1928] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:43.529732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463979c-1d3e-41dd-aefc-d11f9bc8f181 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.531663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1463979c-1d3e-41dd-aefc-d11f9bc8f181 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:43.536890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 953/1929] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:43.544257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1463979c-1d3e-41dd-aefc-d11f9bc8f181 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.545660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e6b9-72c5-4987-a9b8-8e0aa0c19f06 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.546035 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e6b9-72c5-4987-a9b8-8e0aa0c19f06 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.546463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9e6b9-72c5-4987-a9b8-8e0aa0c19f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.549162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c37ca6-af3d-4c0d-9559-172ed77e807b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.549819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37ca6-af3d-4c0d-9559-172ed77e807b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.550130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37ca6-af3d-4c0d-9559-172ed77e807b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.550401 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c37ca6-af3d-4c0d-9559-172ed77e807b admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.551127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c37ca6-af3d-4c0d-9559-172ed77e807b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.552978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c37ca6-af3d-4c0d-9559-172ed77e807b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.566627 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 977/1930] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:43.567957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cad9e6b9-72c5-4987-a9b8-8e0aa0c19f06 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:43.569353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cad9e6b9-72c5-4987-a9b8-8e0aa0c19f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:43.570412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c37ca6-af3d-4c0d-9559-172ed77e807b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.582882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c37ca6-af3d-4c0d-9559-172ed77e807b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.583590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 954/1931] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:43.585234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9e6b9-72c5-4987-a9b8-8e0aa0c19f06 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.585922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7d428-7628-4612-84ab-7a9ce7e9dc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.586710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7d428-7628-4612-84ab-7a9ce7e9dc56 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.587089 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7d428-7628-4612-84ab-7a9ce7e9dc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.587463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa7d428-7628-4612-84ab-7a9ce7e9dc56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.588284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7d428-7628-4612-84ab-7a9ce7e9dc56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.589958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa7d428-7628-4612-84ab-7a9ce7e9dc56 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.597004 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9e6b9-72c5-4987-a9b8-8e0aa0c19f06 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.602263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6db26-3e2f-4938-9b9f-f42598254b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.602444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 978/1932] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:43.603340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6db26-3e2f-4938-9b9f-f42598254b3f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.603631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6db26-3e2f-4938-9b9f-f42598254b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.603944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f6db26-3e2f-4938-9b9f-f42598254b3f admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.605192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6db26-3e2f-4938-9b9f-f42598254b3f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:43.605516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6db26-3e2f-4938-9b9f-f42598254b3f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.606543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7d428-7628-4612-84ab-7a9ce7e9dc56 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.607230 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f6db26-3e2f-4938-9b9f-f42598254b3f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.617601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa7d428-7628-4612-84ab-7a9ce7e9dc56 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.623283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ec9cf8-6151-4d6f-9d77-ffb47878758f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.625168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec9cf8-6151-4d6f-9d77-ffb47878758f admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.625282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 955/1933] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:05:43.625731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec9cf8-6151-4d6f-9d77-ffb47878758f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.626183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6db26-3e2f-4938-9b9f-f42598254b3f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.626419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ec9cf8-6151-4d6f-9d77-ffb47878758f admin admin] PATH_INFO: `/v3/projects/1e229aefaff54691854326c4fef8072c/users/fd788c15bcf4474984c15777e924c831/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.638475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f6db26-3e2f-4938-9b9f-f42598254b3f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.643350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec9cf8-6151-4d6f-9d77-ffb47878758f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e229aefaff54691854326c4fef8072c, user_id=fd788c15bcf4474984c15777e924c831, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.644037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115010a4-0b6f-4fc8-a419-17647dd31201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.644705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115010a4-0b6f-4fc8-a419-17647dd31201 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.644993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115010a4-0b6f-4fc8-a419-17647dd31201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.645273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115010a4-0b6f-4fc8-a419-17647dd31201 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.645971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115010a4-0b6f-4fc8-a419-17647dd31201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.647234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ec9cf8-6151-4d6f-9d77-ffb47878758f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.647900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115010a4-0b6f-4fc8-a419-17647dd31201 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.664772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 979/1934] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:43 2022] PUT /identity/v3/projects/1e229aefaff54691854326c4fef8072c/users/fd788c15bcf4474984c15777e924c831/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:43.676766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f91d1-045b-4252-85ad-c624b2d754a9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.677127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f91d1-045b-4252-85ad-c624b2d754a9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.677545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3f91d1-045b-4252-85ad-c624b2d754a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.682360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 956/1935] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:43.683413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115010a4-0b6f-4fc8-a419-17647dd31201 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.711803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5b3f91d1-045b-4252-85ad-c624b2d754a9 None None] MFA Rules not processed for user `fd788c15bcf4474984c15777e924c831`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:43.713027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5b3f91d1-045b-4252-85ad-c624b2d754a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:43.753785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115010a4-0b6f-4fc8-a419-17647dd31201 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.759024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7e287-bec7-4738-928c-0ac46331f4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.759701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7e287-bec7-4738-928c-0ac46331f4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.760024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7e287-bec7-4738-928c-0ac46331f4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.760305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d7e287-bec7-4738-928c-0ac46331f4c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.761022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7e287-bec7-4738-928c-0ac46331f4c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.762972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d7e287-bec7-4738-928c-0ac46331f4c4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.764109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:43.784036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 980/1936] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:43.784384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 957/1937] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/users => generated 282 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:43.788320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7e287-bec7-4738-928c-0ac46331f4c4 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.788455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f91d1-045b-4252-85ad-c624b2d754a9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.798585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d7e287-bec7-4738-928c-0ac46331f4c4 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.799291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3f91d1-045b-4252-85ad-c624b2d754a9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.803481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d1333-fd85-44e1-a58f-cde7348d6287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.804107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e60ffe-c063-4fcb-9143-01edc9a09206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.804248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d1333-fd85-44e1-a58f-cde7348d6287 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.804423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d1333-fd85-44e1-a58f-cde7348d6287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.804678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d1333-fd85-44e1-a58f-cde7348d6287 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.804862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e60ffe-c063-4fcb-9143-01edc9a09206 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.805081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e60ffe-c063-4fcb-9143-01edc9a09206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.805376 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e60ffe-c063-4fcb-9143-01edc9a09206 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.805475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d1333-fd85-44e1-a58f-cde7348d6287 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.806132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e60ffe-c063-4fcb-9143-01edc9a09206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.807401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049d1333-fd85-44e1-a58f-cde7348d6287 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.807703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e60ffe-c063-4fcb-9143-01edc9a09206 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.818250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 981/1938] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:43.822023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e60ffe-c063-4fcb-9143-01edc9a09206 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.831533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e60ffe-c063-4fcb-9143-01edc9a09206 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.833483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 958/1939] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:43.837366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d1333-fd85-44e1-a58f-cde7348d6287 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.894704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa15ee-d92e-4f66-96fc-1ce8fa0ba4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.895374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa15ee-d92e-4f66-96fc-1ce8fa0ba4e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.895662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa15ee-d92e-4f66-96fc-1ce8fa0ba4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.895944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa15ee-d92e-4f66-96fc-1ce8fa0ba4e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.896678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa15ee-d92e-4f66-96fc-1ce8fa0ba4e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.898280 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa15ee-d92e-4f66-96fc-1ce8fa0ba4e4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.906920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d1333-fd85-44e1-a58f-cde7348d6287 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.911583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f70cff-d248-464c-b5c7-f0eb6626bd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.912257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f70cff-d248-464c-b5c7-f0eb6626bd44 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.912546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f70cff-d248-464c-b5c7-f0eb6626bd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.912827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f70cff-d248-464c-b5c7-f0eb6626bd44 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.913551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f70cff-d248-464c-b5c7-f0eb6626bd44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.915458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 982/1940] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:43.918136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f70cff-d248-464c-b5c7-f0eb6626bd44 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.919564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa15ee-d92e-4f66-96fc-1ce8fa0ba4e4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.929389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa15ee-d92e-4f66-96fc-1ce8fa0ba4e4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.934196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b9890-c83a-44b6-aaf2-2b99e8e879e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.935060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b9890-c83a-44b6-aaf2-2b99e8e879e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.935348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b9890-c83a-44b6-aaf2-2b99e8e879e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.935627 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b9890-c83a-44b6-aaf2-2b99e8e879e4 admin admin] PATH_INFO: `/v3/projects/3d33e8ab641a484098edc82d52d981f8/users/8cbf7c1d4562413b882ea7c9cbaa4b58/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.939343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 959/1941] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:43.943993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f70cff-d248-464c-b5c7-f0eb6626bd44 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.955411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f70cff-d248-464c-b5c7-f0eb6626bd44 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:43.955832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b9890-c83a-44b6-aaf2-2b99e8e879e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d33e8ab641a484098edc82d52d981f8, user_id=8cbf7c1d4562413b882ea7c9cbaa4b58, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.958732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b9890-c83a-44b6-aaf2-2b99e8e879e4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.961185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ebc9a-9d5e-4ce7-b918-d8e1d92351af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:43.962094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ebc9a-9d5e-4ce7-b918-d8e1d92351af admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.962261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ebc9a-9d5e-4ce7-b918-d8e1d92351af admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.962546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ebc9a-9d5e-4ce7-b918-d8e1d92351af admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:43.963440 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ebc9a-9d5e-4ce7-b918-d8e1d92351af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:43.965313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87ebc9a-9d5e-4ce7-b918-d8e1d92351af admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:43.967054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:43.985866 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 983/1942] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:43 2022] PUT /identity/v3/projects/3d33e8ab641a484098edc82d52d981f8/users/8cbf7c1d4562413b882ea7c9cbaa4b58/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:43.986073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 960/1943] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:43.986186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b9890-c83a-44b6-aaf2-2b99e8e879e4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:43.992755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c6958-0b48-4217-8f63-9acd2ff828ec None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:43.994598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c6958-0b48-4217-8f63-9acd2ff828ec None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:43.994750 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c6958-0b48-4217-8f63-9acd2ff828ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:44.042384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-127c6958-0b48-4217-8f63-9acd2ff828ec None None] MFA Rules not processed for user `8cbf7c1d4562413b882ea7c9cbaa4b58`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:44.044315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-127c6958-0b48-4217-8f63-9acd2ff828ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:44.152365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 961/1944] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:44.157877 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c6958-0b48-4217-8f63-9acd2ff828ec None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:44.170131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b9890-c83a-44b6-aaf2-2b99e8e879e4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:44.177760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127c6958-0b48-4217-8f63-9acd2ff828ec None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:44.192428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273fed3-67e7-412e-806d-cd1d6d6c3c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:44.193913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273fed3-67e7-412e-806d-cd1d6d6c3c69 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:44.194162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7dbab-d30c-483f-9954-27c5829fc70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:44.195078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7dbab-d30c-483f-9954-27c5829fc70c admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:44.195192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7dbab-d30c-483f-9954-27c5829fc70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:44.195400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7dbab-d30c-483f-9954-27c5829fc70c admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:44.195897 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273fed3-67e7-412e-806d-cd1d6d6c3c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:44.196102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7dbab-d30c-483f-9954-27c5829fc70c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:44.197222 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4273fed3-67e7-412e-806d-cd1d6d6c3c69 admin admin] PATH_INFO: `/v3/projects/581619048e684fa3a55b263b9fceeffb/users/acde308ce54a4631ad875c3c22ad83fb/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:44.198019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b7dbab-d30c-483f-9954-27c5829fc70c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:44.244984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273fed3-67e7-412e-806d-cd1d6d6c3c69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=581619048e684fa3a55b263b9fceeffb, user_id=acde308ce54a4631ad875c3c22ad83fb, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:44.255336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4273fed3-67e7-412e-806d-cd1d6d6c3c69 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:44.262162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 962/1945] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:44 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:44.263905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7dbab-d30c-483f-9954-27c5829fc70c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:44.285794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 984/1946] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:43 2022] PUT /identity/v3/projects/581619048e684fa3a55b263b9fceeffb/users/acde308ce54a4631ad875c3c22ad83fb/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:44.289767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273fed3-67e7-412e-806d-cd1d6d6c3c69 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:44.375256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:44.407250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b7dbab-d30c-483f-9954-27c5829fc70c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:44.431409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4273fed3-67e7-412e-806d-cd1d6d6c3c69 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:44.446554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e64a5-1feb-4b30-a3cf-d9ddef754eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:44.447358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e64a5-1feb-4b30-a3cf-d9ddef754eac admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:44.447653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e64a5-1feb-4b30-a3cf-d9ddef754eac admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:44.447930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e64a5-1feb-4b30-a3cf-d9ddef754eac admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:44.448723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e64a5-1feb-4b30-a3cf-d9ddef754eac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:44.451268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e64a5-1feb-4b30-a3cf-d9ddef754eac admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:44.467254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 985/1947] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:44.481516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e64a5-1feb-4b30-a3cf-d9ddef754eac admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:44.501670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e64a5-1feb-4b30-a3cf-d9ddef754eac admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:44.509957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4af7b4-1859-4ab9-b2bb-4e9359a1bec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:44.511065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4af7b4-1859-4ab9-b2bb-4e9359a1bec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:44.511531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4af7b4-1859-4ab9-b2bb-4e9359a1bec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:44.511915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4af7b4-1859-4ab9-b2bb-4e9359a1bec1 admin admin] PATH_INFO: `/v3/projects/581619048e684fa3a55b263b9fceeffb/users/acde308ce54a4631ad875c3c22ad83fb/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:44.516218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4af7b4-1859-4ab9-b2bb-4e9359a1bec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=581619048e684fa3a55b263b9fceeffb, user_id=acde308ce54a4631ad875c3c22ad83fb, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:44.520641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4af7b4-1859-4ab9-b2bb-4e9359a1bec1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:44.534230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 986/1948] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:44 2022] PUT /identity/v3/projects/581619048e684fa3a55b263b9fceeffb/users/acde308ce54a4631ad875c3c22ad83fb/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:44.544341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa05678-a1e1-4789-8543-d6822b09ca96 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:44.545478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa05678-a1e1-4789-8543-d6822b09ca96 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:44.545478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa05678-a1e1-4789-8543-d6822b09ca96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:44.592244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a853d9-836d-4a57-bbaf-d0132e2963cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:44.592856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a853d9-836d-4a57-bbaf-d0132e2963cc admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:44.592856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a853d9-836d-4a57-bbaf-d0132e2963cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:44.593177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a853d9-836d-4a57-bbaf-d0132e2963cc admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:44.594120 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a853d9-836d-4a57-bbaf-d0132e2963cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:44.594304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-afa05678-a1e1-4789-8543-d6822b09ca96 None None] MFA Rules not processed for user `acde308ce54a4631ad875c3c22ad83fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:44.595776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-afa05678-a1e1-4789-8543-d6822b09ca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:44.596124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a853d9-836d-4a57-bbaf-d0132e2963cc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:44.615086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:44.622819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 963/1949] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:44 2022] POST /identity/v3/users => generated 288 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:44.626669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a853d9-836d-4a57-bbaf-d0132e2963cc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:44.651544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a853d9-836d-4a57-bbaf-d0132e2963cc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:44.694091 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 987/1950] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:44 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:44.705599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa05678-a1e1-4789-8543-d6822b09ca96 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:44.716186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-afa05678-a1e1-4789-8543-d6822b09ca96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:44.764609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310545e-d4be-4f87-8e8d-dd6c3f102a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:44.768293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310545e-d4be-4f87-8e8d-dd6c3f102a17 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:44.768747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310545e-d4be-4f87-8e8d-dd6c3f102a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:44.768881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4310545e-d4be-4f87-8e8d-dd6c3f102a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:44.771858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310545e-d4be-4f87-8e8d-dd6c3f102a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:44.776251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4310545e-d4be-4f87-8e8d-dd6c3f102a17 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:44.803669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 964/1951] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:44.808177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310545e-d4be-4f87-8e8d-dd6c3f102a17 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:44.812826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa05678-a1e1-4789-8543-d6822b09ca96 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:44.821278 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564295f-d9da-47aa-8bd5-4b2d2b99fa18 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:44.822280 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564295f-d9da-47aa-8bd5-4b2d2b99fa18 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:44.822431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4310545e-d4be-4f87-8e8d-dd6c3f102a17 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:44.822817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564295f-d9da-47aa-8bd5-4b2d2b99fa18 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:44.823700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564295f-d9da-47aa-8bd5-4b2d2b99fa18 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:44.828143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8051-eece-449b-8d76-e8c23e3040e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:44.829222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8051-eece-449b-8d76-e8c23e3040e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:44.829492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8051-eece-449b-8d76-e8c23e3040e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:44.829846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd8051-eece-449b-8d76-e8c23e3040e0 admin admin] PATH_INFO: `/v3/projects/fc3d03c958534f7f87a7c7a847e34735/users/b59389232e3e4952a36e80ffa86f2d60/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:44.838254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564295f-d9da-47aa-8bd5-4b2d2b99fa18 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:44.840559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6564295f-d9da-47aa-8bd5-4b2d2b99fa18 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:44.847361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd8051-eece-449b-8d76-e8c23e3040e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3d03c958534f7f87a7c7a847e34735, user_id=b59389232e3e4952a36e80ffa86f2d60, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:44.850548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd8051-eece-449b-8d76-e8c23e3040e0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:44.871945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 988/1952] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:44 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:44.885075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 965/1953] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:44 2022] PUT /identity/v3/projects/fc3d03c958534f7f87a7c7a847e34735/users/b59389232e3e4952a36e80ffa86f2d60/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:44.889528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564295f-d9da-47aa-8bd5-4b2d2b99fa18 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:44.901585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564295f-d9da-47aa-8bd5-4b2d2b99fa18 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:44.916813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8051-eece-449b-8d76-e8c23e3040e0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:44.918839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f4bd8051-eece-449b-8d76-e8c23e3040e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:44.968543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:44.995482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd8051-eece-449b-8d76-e8c23e3040e0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:44.997304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed4d99-2020-46bd-8441-25ee2fc83918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:45.000356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f83e4-0c2f-4ffd-bacb-0c7b0f2c0085 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:45.001945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f83e4-0c2f-4ffd-bacb-0c7b0f2c0085 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:45.002477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f83e4-0c2f-4ffd-bacb-0c7b0f2c0085 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:45.002943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2f83e4-0c2f-4ffd-bacb-0c7b0f2c0085 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:45.005368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4b2f83e4-0c2f-4ffd-bacb-0c7b0f2c0085 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:45.009682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed4d99-2020-46bd-8441-25ee2fc83918 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:45.010859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed4d99-2020-46bd-8441-25ee2fc83918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:45.011220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fed4d99-2020-46bd-8441-25ee2fc83918 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:45.012917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed4d99-2020-46bd-8441-25ee2fc83918 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:45.016257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fed4d99-2020-46bd-8441-25ee2fc83918 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:45.037690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 989/1954] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:45.042205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed4d99-2020-46bd-8441-25ee2fc83918 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:45.053870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fed4d99-2020-46bd-8441-25ee2fc83918 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:45.059815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da689d-4d77-4791-9a1d-a7028e960057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:45.060858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da689d-4d77-4791-9a1d-a7028e960057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:45.061144 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da689d-4d77-4791-9a1d-a7028e960057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:45.061782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da689d-4d77-4791-9a1d-a7028e960057 admin admin] PATH_INFO: `/v3/projects/fc3d03c958534f7f87a7c7a847e34735/users/b59389232e3e4952a36e80ffa86f2d60/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:45.064041 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da689d-4d77-4791-9a1d-a7028e960057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc3d03c958534f7f87a7c7a847e34735, user_id=b59389232e3e4952a36e80ffa86f2d60, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:45.066981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99da689d-4d77-4791-9a1d-a7028e960057 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:45.081526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 990/1955] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:45 2022] PUT /identity/v3/projects/fc3d03c958534f7f87a7c7a847e34735/users/b59389232e3e4952a36e80ffa86f2d60/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:45.084989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f83e4-0c2f-4ffd-bacb-0c7b0f2c0085 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:45.086579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2f83e4-0c2f-4ffd-bacb-0c7b0f2c0085 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:45.091843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9004-5254-4231-a0b0-69c0db6abbae None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:45.092134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9004-5254-4231-a0b0-69c0db6abbae None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:45.092413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b9004-5254-4231-a0b0-69c0db6abbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:45.100082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 966/1956] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:44 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:45.131808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ce0b9004-5254-4231-a0b0-69c0db6abbae None None] MFA Rules not processed for user `b59389232e3e4952a36e80ffa86f2d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:45.133564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ce0b9004-5254-4231-a0b0-69c0db6abbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:45.229700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 991/1957] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:45 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:45.241680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f83e4-0c2f-4ffd-bacb-0c7b0f2c0085 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:45.250572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2f83e4-0c2f-4ffd-bacb-0c7b0f2c0085 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:45.290603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:45.323959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1617b2-e5cc-43c3-8651-6c057cfaf604 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:45.324724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1617b2-e5cc-43c3-8651-6c057cfaf604 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:45.325086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1617b2-e5cc-43c3-8651-6c057cfaf604 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:45.325366 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1617b2-e5cc-43c3-8651-6c057cfaf604 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:45.337466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1617b2-e5cc-43c3-8651-6c057cfaf604 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:45.340999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1617b2-e5cc-43c3-8651-6c057cfaf604 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:45.351691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 967/1958] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:45 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:45.434464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b9004-5254-4231-a0b0-69c0db6abbae None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:45.461630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b9004-5254-4231-a0b0-69c0db6abbae None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:45.465258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5587c7-1ca4-46bd-b563-4d9a6352c71b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:45.466318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5587c7-1ca4-46bd-b563-4d9a6352c71b service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:45.466601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5587c7-1ca4-46bd-b563-4d9a6352c71b service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:45.466877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5587c7-1ca4-46bd-b563-4d9a6352c71b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:45.558097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5587c7-1ca4-46bd-b563-4d9a6352c71b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:45.559551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5587c7-1ca4-46bd-b563-4d9a6352c71b service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:45.571877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 992/1959] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:45 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:47.397102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1617b2-e5cc-43c3-8651-6c057cfaf604 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:47.398904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4d1617b2-e5cc-43c3-8651-6c057cfaf604 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:47.491991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1617b2-e5cc-43c3-8651-6c057cfaf604 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:47.496870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:47.497718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:47.497998 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:47.498702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] PATH_INFO: `/v3/users/f45e05b1ceb34bbbbf79fee531174707` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:47.499941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f45e05b1ceb34bbbbf79fee531174707)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:47.501997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:47.564114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f45e05b1ceb34bbbbf79fee531174707'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:47.565185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f45e05b1ceb34bbbbf79fee531174707'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:47.565479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f45e05b1ceb34bbbbf79fee531174707'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:47.580708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f45e05b1ceb34bbbbf79fee531174707'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:47.594079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f45e05b1ceb34bbbbf79fee531174707'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:47.614097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 968/1960] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:47 2022] DELETE /identity/v3/users/f45e05b1ceb34bbbbf79fee531174707 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:47.623109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cc2db-005f-487b-8c50-10f068c62277 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:47.623618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cc2db-005f-487b-8c50-10f068c62277 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:47.623996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cc2db-005f-487b-8c50-10f068c62277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:47.641820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0f3cc2db-005f-487b-8c50-10f068c62277 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:47.643053 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0f3cc2db-005f-487b-8c50-10f068c62277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:47.694718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:47.709890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 993/1961] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:47.724260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:47.725837 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:47.783665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:47.788032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ede832-7ed9-4986-9303-1d7c80681d10 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:47.793170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a72da-7cb3-4c7f-81f9-0c4554b7cad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:47.793835 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a72da-7cb3-4c7f-81f9-0c4554b7cad8 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:47.794140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a72da-7cb3-4c7f-81f9-0c4554b7cad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:47.794427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2a72da-7cb3-4c7f-81f9-0c4554b7cad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:47.804272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a72da-7cb3-4c7f-81f9-0c4554b7cad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:47.805988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2a72da-7cb3-4c7f-81f9-0c4554b7cad8 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:47.818802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 969/1962] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:48.144703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cc2db-005f-487b-8c50-10f068c62277 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:48.146400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0f3cc2db-005f-487b-8c50-10f068c62277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:48.160898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cc2db-005f-487b-8c50-10f068c62277 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:48.173967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:48.175026 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:48.175314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:48.175595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] PATH_INFO: `/v3/projects/90f7307168c24bd1aef024ca8984c16e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:48.176995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90f7307168c24bd1aef024ca8984c16e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:48.178976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:48.246630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90f7307168c24bd1aef024ca8984c16e', 'request_id': 'req-fb3b0b57-5a71-448f-af47-a4da2bc46c42'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.260245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90f7307168c24bd1aef024ca8984c16e', 'request_id': 'req-fb3b0b57-5a71-448f-af47-a4da2bc46c42'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.275631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90f7307168c24bd1aef024ca8984c16e', 'request_id': 'req-fb3b0b57-5a71-448f-af47-a4da2bc46c42'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.293950 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] The token cache is being invalidate because project 90f7307168c24bd1aef024ca8984c16e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:48.294563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.295853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.299422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 994/1963] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:48 2022] DELETE /identity/v3/projects/90f7307168c24bd1aef024ca8984c16e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:48.303575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a72da-7cb3-4c7f-81f9-0c4554b7cad8 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:48.319755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:48.319910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2a72da-7cb3-4c7f-81f9-0c4554b7cad8 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:48.321432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:48.406965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:48.407997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:48.408285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:48.408592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] PATH_INFO: `/v3/users/291496959b2747d7b62ebcca3f413511` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:48.410584 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=291496959b2747d7b62ebcca3f413511)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:48.412459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:48.413745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b0b57-5a71-448f-af47-a4da2bc46c42 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:48.421658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:48.422616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:48.422905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:48.423188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] PATH_INFO: `/v3/users/132f18d29706457dadc37ccbc9f2dc75` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:48.424440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=132f18d29706457dadc37ccbc9f2dc75)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:48.427251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:48.480141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '291496959b2747d7b62ebcca3f413511'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.481154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '291496959b2747d7b62ebcca3f413511'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.481993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '291496959b2747d7b62ebcca3f413511'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.501662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '291496959b2747d7b62ebcca3f413511'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.506630 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '132f18d29706457dadc37ccbc9f2dc75'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.508408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '132f18d29706457dadc37ccbc9f2dc75'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.514284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a8d4e56c-91ab-4032-adf8-d72532593d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '291496959b2747d7b62ebcca3f413511'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.517305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '132f18d29706457dadc37ccbc9f2dc75'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.531813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '132f18d29706457dadc37ccbc9f2dc75'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.532595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '132f18d29706457dadc37ccbc9f2dc75'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.534885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 970/1964] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:48 2022] DELETE /identity/v3/users/291496959b2747d7b62ebcca3f413511 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:48.554130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:48.569670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 995/1965] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:48 2022] DELETE /identity/v3/users/132f18d29706457dadc37ccbc9f2dc75 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:48.574097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38523f7-b173-48de-a236-b84b828e3e6f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:48.574610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38523f7-b173-48de-a236-b84b828e3e6f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:48.575087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38523f7-b173-48de-a236-b84b828e3e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:48.602734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a38523f7-b173-48de-a236-b84b828e3e6f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:48.604619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a38523f7-b173-48de-a236-b84b828e3e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:48.614707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:48.616834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:48.734451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 971/1966] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:48.745288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38523f7-b173-48de-a236-b84b828e3e6f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:48.747298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a38523f7-b173-48de-a236-b84b828e3e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:48.752814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af85f6b-2926-4ea5-8270-cff6c9b44f6c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:48.758385 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:48.759350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:48.759644 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:48.759924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] PATH_INFO: `/v3/users/8cbf7c1d4562413b882ea7c9cbaa4b58` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:48.761226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbf7c1d4562413b882ea7c9cbaa4b58)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:48.763297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:48.764554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:48.824915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38523f7-b173-48de-a236-b84b828e3e6f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:48.830233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7db4a22-80a6-411e-9398-235246049eef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:48.830996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db4a22-80a6-411e-9398-235246049eef service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:48.831408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db4a22-80a6-411e-9398-235246049eef service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:48.831771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7db4a22-80a6-411e-9398-235246049eef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:48.853324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbf7c1d4562413b882ea7c9cbaa4b58'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.854391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbf7c1d4562413b882ea7c9cbaa4b58'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.860531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db4a22-80a6-411e-9398-235246049eef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:48.862671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7db4a22-80a6-411e-9398-235246049eef service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:48.877945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbf7c1d4562413b882ea7c9cbaa4b58'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.880414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 972/1967] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:05:48.890601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbf7c1d4562413b882ea7c9cbaa4b58'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.891502 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbf7c1d4562413b882ea7c9cbaa4b58'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:48.919770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 996/1968] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:48 2022] DELETE /identity/v3/users/8cbf7c1d4562413b882ea7c9cbaa4b58 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:48.924228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26836c94-44c9-47f2-97dc-21a583336370 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:48.924524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26836c94-44c9-47f2-97dc-21a583336370 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:48.924810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26836c94-44c9-47f2-97dc-21a583336370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:48.950284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-26836c94-44c9-47f2-97dc-21a583336370 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:48.951607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-26836c94-44c9-47f2-97dc-21a583336370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:49.105632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 973/1969] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:49.125521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:49.127242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:49.133655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26836c94-44c9-47f2-97dc-21a583336370 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:49.136318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-26836c94-44c9-47f2-97dc-21a583336370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:49.164171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26836c94-44c9-47f2-97dc-21a583336370 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:49.176760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:49.177706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:49.177996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:49.178310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] PATH_INFO: `/v3/projects/c627a339d26c4bf594ab064ba7df11d6` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:49.180706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c627a339d26c4bf594ab064ba7df11d6)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:49.182658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:49.237349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397594d-518f-4f5a-b478-0b7f6c075ff7 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:49.241184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e808456-1370-4a37-af34-f3675095f53b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:49.241863 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e808456-1370-4a37-af34-f3675095f53b service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:49.242187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e808456-1370-4a37-af34-f3675095f53b service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:49.242464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e808456-1370-4a37-af34-f3675095f53b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:49.255078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e808456-1370-4a37-af34-f3675095f53b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:49.256529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e808456-1370-4a37-af34-f3675095f53b service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:49.268058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c627a339d26c4bf594ab064ba7df11d6', 'request_id': 'req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.269075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 997/1970] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:49.277322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c627a339d26c4bf594ab064ba7df11d6', 'request_id': 'req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.286539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c627a339d26c4bf594ab064ba7df11d6', 'request_id': 'req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.300590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] The token cache is being invalidate because project c627a339d26c4bf594ab064ba7df11d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:49.301242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.302128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.304476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 974/1971] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:49 2022] DELETE /identity/v3/projects/c627a339d26c4bf594ab064ba7df11d6 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:49.308476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e808456-1370-4a37-af34-f3675095f53b service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:49.320055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e808456-1370-4a37-af34-f3675095f53b service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:49.338300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:49.422209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:49.423619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:49.424580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:49.424874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] PATH_INFO: `/v3/users/026587500988433a80af47f20940e009` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:49.427165 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=026587500988433a80af47f20940e009)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:49.430827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:49.528180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:49.529749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:49.554354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '026587500988433a80af47f20940e009'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.555239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f4088-bdfe-4b96-bd76-e9bdd93e290a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:49.555420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '026587500988433a80af47f20940e009'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.560503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:49.561272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:49.561600 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:49.561854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] PATH_INFO: `/v3/projects/ab1e05f63e5d449f91ace0121205c1ea` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:49.563238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab1e05f63e5d449f91ace0121205c1ea)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:49.565491 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:49.569145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '026587500988433a80af47f20940e009'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.582421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '026587500988433a80af47f20940e009'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.583171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '026587500988433a80af47f20940e009'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.612509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:49.624102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 998/1972] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:49 2022] DELETE /identity/v3/users/026587500988433a80af47f20940e009 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:49.639928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab1e05f63e5d449f91ace0121205c1ea', 'request_id': 'req-defbe9f4-21ed-4e63-bfc0-440831c609ef'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.651650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab1e05f63e5d449f91ace0121205c1ea', 'request_id': 'req-defbe9f4-21ed-4e63-bfc0-440831c609ef'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.659292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab1e05f63e5d449f91ace0121205c1ea', 'request_id': 'req-defbe9f4-21ed-4e63-bfc0-440831c609ef'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.672747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] The token cache is being invalidate because project ab1e05f63e5d449f91ace0121205c1ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:49.673604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.674592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.676623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 975/1973] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:49 2022] DELETE /identity/v3/projects/ab1e05f63e5d449f91ace0121205c1ea => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:49.680972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:49.682553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:49.759136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d35bc2-19a5-4c8c-ba0d-7ee9a8de9a26 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:49.764719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:49.766200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:49.766487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:49.766763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] PATH_INFO: `/v3/users/fd788c15bcf4474984c15777e924c831` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:49.768167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd788c15bcf4474984c15777e924c831)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:49.769359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:49.769922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:49.771245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:49.831414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:49.839556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd788c15bcf4474984c15777e924c831'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.841213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd788c15bcf4474984c15777e924c831'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.855966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd788c15bcf4474984c15777e924c831'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.861134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defbe9f4-21ed-4e63-bfc0-440831c609ef admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:49.872831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:49.873861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:49.874000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:49.874236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] PATH_INFO: `/v3/projects/3d33e8ab641a484098edc82d52d981f8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:49.877203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d33e8ab641a484098edc82d52d981f8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:49.879312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd788c15bcf4474984c15777e924c831'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.879570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:49.880157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd788c15bcf4474984c15777e924c831'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.905771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 999/1974] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:49 2022] DELETE /identity/v3/users/fd788c15bcf4474984c15777e924c831 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:49.937849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d33e8ab641a484098edc82d52d981f8', 'request_id': 'req-e4fdbed5-99d0-463b-a6c3-434b6915d916'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.945614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d33e8ab641a484098edc82d52d981f8', 'request_id': 'req-e4fdbed5-99d0-463b-a6c3-434b6915d916'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.952021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d33e8ab641a484098edc82d52d981f8', 'request_id': 'req-e4fdbed5-99d0-463b-a6c3-434b6915d916'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.963551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] The token cache is being invalidate because project 3d33e8ab641a484098edc82d52d981f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:49.963920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.964586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:49.966390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 976/1975] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:49 2022] DELETE /identity/v3/projects/3d33e8ab641a484098edc82d52d981f8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:49.970291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:49.971808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:50.054985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:50.058386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:50.088976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047e0ee-9f9c-4339-b489-3be4e93523a9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:50.181220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:50.182521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:50.183861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:50.184525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] PATH_INFO: `/v3/users/b59389232e3e4952a36e80ffa86f2d60` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:50.188762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b59389232e3e4952a36e80ffa86f2d60)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:50.192438 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:50.216625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:50.253682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fdbed5-99d0-463b-a6c3-434b6915d916 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:50.264457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:50.266945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:50.267217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b59389232e3e4952a36e80ffa86f2d60'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.267780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:50.268143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] PATH_INFO: `/v3/projects/bfbe57febb0344348af2dd5d6283fb75` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:50.268760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b59389232e3e4952a36e80ffa86f2d60'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.271652 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfbe57febb0344348af2dd5d6283fb75)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:50.275506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:50.278158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b59389232e3e4952a36e80ffa86f2d60'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.313575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b59389232e3e4952a36e80ffa86f2d60'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.314631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b59389232e3e4952a36e80ffa86f2d60'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.317397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:50.363854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1000/1976] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:49 2022] DELETE /identity/v3/users/b59389232e3e4952a36e80ffa86f2d60 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:50.369378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfbe57febb0344348af2dd5d6283fb75', 'request_id': 'req-97953279-9ff7-4a49-b5e4-bff3fc938f64'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.376114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:50.378691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:50.380554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfbe57febb0344348af2dd5d6283fb75', 'request_id': 'req-97953279-9ff7-4a49-b5e4-bff3fc938f64'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.387777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfbe57febb0344348af2dd5d6283fb75', 'request_id': 'req-97953279-9ff7-4a49-b5e4-bff3fc938f64'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.400015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] The token cache is being invalidate because project bfbe57febb0344348af2dd5d6283fb75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:50.400445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.401212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-97953279-9ff7-4a49-b5e4-bff3fc938f64 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.407698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 977/1977] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:50 2022] DELETE /identity/v3/projects/bfbe57febb0344348af2dd5d6283fb75 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:50.409250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f7175-acf4-43e3-a06d-7d5c41a9451a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:50.409621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f7175-acf4-43e3-a06d-7d5c41a9451a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:50.409968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8f7175-acf4-43e3-a06d-7d5c41a9451a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:50.459858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9f8f7175-acf4-43e3-a06d-7d5c41a9451a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:50.462377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9f8f7175-acf4-43e3-a06d-7d5c41a9451a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:50.498947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d54f9-9576-4391-ada1-d69f19a3f819 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:50.508607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:50.509144 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:50.509429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:50.509706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] PATH_INFO: `/v3/projects/1e229aefaff54691854326c4fef8072c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:50.511908 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e229aefaff54691854326c4fef8072c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:50.514087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:50.557752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 978/1978] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:50.565678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e538120-8ef1-428f-ab2b-73458aabe414 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:50.565993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e538120-8ef1-428f-ab2b-73458aabe414 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:50.566285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e538120-8ef1-428f-ab2b-73458aabe414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:50.590581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7e538120-8ef1-428f-ab2b-73458aabe414 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:50.591812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7e538120-8ef1-428f-ab2b-73458aabe414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:50.594526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e229aefaff54691854326c4fef8072c', 'request_id': 'req-1d174fb5-5742-4a32-9063-fe1c662fc3e3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.600922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e229aefaff54691854326c4fef8072c', 'request_id': 'req-1d174fb5-5742-4a32-9063-fe1c662fc3e3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.613341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e229aefaff54691854326c4fef8072c', 'request_id': 'req-1d174fb5-5742-4a32-9063-fe1c662fc3e3'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.630981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] The token cache is being invalidate because project 1e229aefaff54691854326c4fef8072c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:50.631272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.631970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.645245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1001/1979] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:50 2022] DELETE /identity/v3/projects/1e229aefaff54691854326c4fef8072c => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:50.649530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:50.651185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:50.688240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 979/1980] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:50.691474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e538120-8ef1-428f-ab2b-73458aabe414 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:50.706549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e538120-8ef1-428f-ab2b-73458aabe414 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:50.712444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79dbd2-7f9e-499b-a4ad-e7994717cab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:50.713185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79dbd2-7f9e-499b-a4ad-e7994717cab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:50.713474 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79dbd2-7f9e-499b-a4ad-e7994717cab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:50.713873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a79dbd2-7f9e-499b-a4ad-e7994717cab5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:50.714743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79dbd2-7f9e-499b-a4ad-e7994717cab5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:50.715156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79dbd2-7f9e-499b-a4ad-e7994717cab5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:50.718698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a79dbd2-7f9e-499b-a4ad-e7994717cab5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:50.736792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d174fb5-5742-4a32-9063-fe1c662fc3e3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:50.745542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:50.745694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 980/1981] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:05:50.746437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:50.746553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79dbd2-7f9e-499b-a4ad-e7994717cab5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:50.746729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:50.746992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] PATH_INFO: `/v3/users/acde308ce54a4631ad875c3c22ad83fb` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:50.751380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] RBAC: Authorizing `identity:delete_user(user_id=acde308ce54a4631ad875c3c22ad83fb)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:50.755187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:50.764127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a79dbd2-7f9e-499b-a4ad-e7994717cab5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:50.782171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7112b31-656b-4844-b5df-29483300bce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:50.782859 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7112b31-656b-4844-b5df-29483300bce5 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:50.783144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7112b31-656b-4844-b5df-29483300bce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:50.783431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7112b31-656b-4844-b5df-29483300bce5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:50.784134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7112b31-656b-4844-b5df-29483300bce5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:50.786069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7112b31-656b-4844-b5df-29483300bce5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:50.790580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:50.797819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:50.824142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 981/1982] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:50 2022] POST /identity/v3/projects => generated 373 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:50.829639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7112b31-656b-4844-b5df-29483300bce5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:50.855330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acde308ce54a4631ad875c3c22ad83fb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.856125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acde308ce54a4631ad875c3c22ad83fb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.870095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acde308ce54a4631ad875c3c22ad83fb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.876956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acde308ce54a4631ad875c3c22ad83fb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.877835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acde308ce54a4631ad875c3c22ad83fb'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:50.902387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1002/1983] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:50 2022] DELETE /identity/v3/users/acde308ce54a4631ad875c3c22ad83fb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:50.918079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7112b31-656b-4844-b5df-29483300bce5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:51.012731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e00b-6122-460a-b040-25e170f62b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:51.014431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e00b-6122-460a-b040-25e170f62b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.015509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e00b-6122-460a-b040-25e170f62b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.016151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e00b-6122-460a-b040-25e170f62b8a admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.017580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387e00b-6122-460a-b040-25e170f62b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:51.020422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a387e00b-6122-460a-b040-25e170f62b8a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:51.043636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 982/1984] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:50 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:51.048285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:51.050493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:51.059285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e00b-6122-460a-b040-25e170f62b8a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:51.068483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a387e00b-6122-460a-b040-25e170f62b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:51.078955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb6f35-8ddd-48db-85d2-af017b8f4495 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:51.091177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e00b-6122-460a-b040-25e170f62b8a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:51.093712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced624c-147c-4282-95c5-7cd2844f521a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:51.094551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced624c-147c-4282-95c5-7cd2844f521a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.094828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced624c-147c-4282-95c5-7cd2844f521a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.095108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced624c-147c-4282-95c5-7cd2844f521a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.095948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced624c-147c-4282-95c5-7cd2844f521a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:51.108362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ced624c-147c-4282-95c5-7cd2844f521a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:51.117939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:51.118844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.119132 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.119412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] PATH_INFO: `/v3/projects/fc3d03c958534f7f87a7c7a847e34735` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.121084 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc3d03c958534f7f87a7c7a847e34735)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:51.124316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:51.136346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1003/1985] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:51.137296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced624c-147c-4282-95c5-7cd2844f521a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:51.165595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced624c-147c-4282-95c5-7cd2844f521a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:51.174733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ecfc0-ec7a-42aa-8e96-2533081917d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:51.175783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ecfc0-ec7a-42aa-8e96-2533081917d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.176274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ecfc0-ec7a-42aa-8e96-2533081917d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.177254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3ecfc0-ec7a-42aa-8e96-2533081917d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.178238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ecfc0-ec7a-42aa-8e96-2533081917d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:51.181134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3ecfc0-ec7a-42aa-8e96-2533081917d2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:51.192253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc3d03c958534f7f87a7c7a847e34735', 'request_id': 'req-894f4b71-e8be-4c9f-a89d-03cd63048d1b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:51.203946 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1004/1986] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:51.205641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ecfc0-ec7a-42aa-8e96-2533081917d2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:51.206390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc3d03c958534f7f87a7c7a847e34735', 'request_id': 'req-894f4b71-e8be-4c9f-a89d-03cd63048d1b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:51.214938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc3d03c958534f7f87a7c7a847e34735', 'request_id': 'req-894f4b71-e8be-4c9f-a89d-03cd63048d1b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:51.221616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3ecfc0-ec7a-42aa-8e96-2533081917d2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:51.232894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] The token cache is being invalidate because project fc3d03c958534f7f87a7c7a847e34735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:51.237547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:51.238532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-894f4b71-e8be-4c9f-a89d-03cd63048d1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:51.245341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 983/1987] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:51 2022] DELETE /identity/v3/projects/fc3d03c958534f7f87a7c7a847e34735 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:51.247138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b452f5-b625-4d36-bb85-ba3047391b58 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.247504 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b452f5-b625-4d36-bb85-ba3047391b58 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.247940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b452f5-b625-4d36-bb85-ba3047391b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.262963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:51.269891 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b5b452f5-b625-4d36-bb85-ba3047391b58 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:51.271471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b5b452f5-b625-4d36-bb85-ba3047391b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:51.332660 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677f6e0a-6176-44b4-93b1-8c0efcd7bad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:51.333796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f6e0a-6176-44b4-93b1-8c0efcd7bad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.335775 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f6e0a-6176-44b4-93b1-8c0efcd7bad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.336831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677f6e0a-6176-44b4-93b1-8c0efcd7bad0 admin admin] PATH_INFO: `/v3/projects/5db904f456b040d8a9f16b169118d21a/users/c31dbb3dcd9a411ea125573c8189e00c/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.373898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 984/1988] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:51.375837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f6e0a-6176-44b4-93b1-8c0efcd7bad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5db904f456b040d8a9f16b169118d21a, user_id=c31dbb3dcd9a411ea125573c8189e00c, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:51.379033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677f6e0a-6176-44b4-93b1-8c0efcd7bad0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:51.384287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a8b58-2876-4386-829c-cc1c1a317fa6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.384588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a8b58-2876-4386-829c-cc1c1a317fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.384875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a8b58-2876-4386-829c-cc1c1a317fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.403921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1005/1989] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:51 2022] PUT /identity/v3/projects/5db904f456b040d8a9f16b169118d21a/users/c31dbb3dcd9a411ea125573c8189e00c/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:51.409074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63351b-c907-4507-9ff1-4cea4ea6126a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.410454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63351b-c907-4507-9ff1-4cea4ea6126a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.410742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63351b-c907-4507-9ff1-4cea4ea6126a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.415719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-345a8b58-2876-4386-829c-cc1c1a317fa6 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:51.417057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-345a8b58-2876-4386-829c-cc1c1a317fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:51.437676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7a63351b-c907-4507-9ff1-4cea4ea6126a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:51.439286 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7a63351b-c907-4507-9ff1-4cea4ea6126a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:51.520926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 985/1990] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:51.524845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a8b58-2876-4386-829c-cc1c1a317fa6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:51.533991 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:51.539805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a8b58-2876-4386-829c-cc1c1a317fa6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:51.557423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1006/1991] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:51.566409 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a986802-176b-43ca-9dc8-f0112483ecf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:51.567128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a986802-176b-43ca-9dc8-f0112483ecf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.567390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a986802-176b-43ca-9dc8-f0112483ecf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.567661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a986802-176b-43ca-9dc8-f0112483ecf9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.568593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a986802-176b-43ca-9dc8-f0112483ecf9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:51.568712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a986802-176b-43ca-9dc8-f0112483ecf9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:51.570318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a986802-176b-43ca-9dc8-f0112483ecf9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:51.587073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 986/1992] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:05:51.588250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63351b-c907-4507-9ff1-4cea4ea6126a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:51.599593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63351b-c907-4507-9ff1-4cea4ea6126a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:51.607181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a986802-176b-43ca-9dc8-f0112483ecf9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:51.607585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42681c9-2e4d-4bce-8bd8-f626d1102fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:51.608399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42681c9-2e4d-4bce-8bd8-f626d1102fe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.610526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42681c9-2e4d-4bce-8bd8-f626d1102fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.610526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42681c9-2e4d-4bce-8bd8-f626d1102fe5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.610719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42681c9-2e4d-4bce-8bd8-f626d1102fe5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:51.620535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42681c9-2e4d-4bce-8bd8-f626d1102fe5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:51.621720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5a986802-176b-43ca-9dc8-f0112483ecf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:51.655389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a986802-176b-43ca-9dc8-f0112483ecf9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:51.665885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:51.666152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1007/1993] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:51 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:51.667021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42681c9-2e4d-4bce-8bd8-f626d1102fe5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:51.668069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.668618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.669113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c42681c9-2e4d-4bce-8bd8-f626d1102fe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:51.669419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] PATH_INFO: `/v3/projects/581619048e684fa3a55b263b9fceeffb` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.671936 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] RBAC: Authorizing `identity:delete_project(project_id=581619048e684fa3a55b263b9fceeffb)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:51.676634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:51.780220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '581619048e684fa3a55b263b9fceeffb', 'request_id': 'req-a4f31378-99d0-4502-be19-59813323279d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:51.791402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42681c9-2e4d-4bce-8bd8-f626d1102fe5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:51.799016 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '581619048e684fa3a55b263b9fceeffb', 'request_id': 'req-a4f31378-99d0-4502-be19-59813323279d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:51.810915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '581619048e684fa3a55b263b9fceeffb', 'request_id': 'req-a4f31378-99d0-4502-be19-59813323279d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:51.830372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] The token cache is being invalidate because project 581619048e684fa3a55b263b9fceeffb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:51.830949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:51.831787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:51.835063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 987/1994] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:51 2022] DELETE /identity/v3/projects/581619048e684fa3a55b263b9fceeffb => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:51.836062 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:51.837769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:51.859873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee97aa21-f4e3-4cf9-bfa3-133e2443a629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:51.860581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97aa21-f4e3-4cf9-bfa3-133e2443a629 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.861126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97aa21-f4e3-4cf9-bfa3-133e2443a629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.861241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee97aa21-f4e3-4cf9-bfa3-133e2443a629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.863371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ee97aa21-f4e3-4cf9-bfa3-133e2443a629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:51.876888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:51.944214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f31378-99d0-4502-be19-59813323279d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:51.952012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f06a0-41f2-4661-a68a-039d509b225a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:51.953478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f06a0-41f2-4661-a68a-039d509b225a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.954058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f06a0-41f2-4661-a68a-039d509b225a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.954646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f06a0-41f2-4661-a68a-039d509b225a admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.955762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f06a0-41f2-4661-a68a-039d509b225a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:51.958194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f06a0-41f2-4661-a68a-039d509b225a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:51.965477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97aa21-f4e3-4cf9-bfa3-133e2443a629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:51.967415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee97aa21-f4e3-4cf9-bfa3-133e2443a629 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:51.982620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1008/1995] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:51.983518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 988/1996] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:51 2022] POST /identity/v3/users => generated 284 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:51.984334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1994aa2-9aba-49e7-a144-e32d61c633cb None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:51.984625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1994aa2-9aba-49e7-a144-e32d61c633cb None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:51.984904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1994aa2-9aba-49e7-a144-e32d61c633cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:51.988059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f06a0-41f2-4661-a68a-039d509b225a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.004458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f06a0-41f2-4661-a68a-039d509b225a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.008699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a1994aa2-9aba-49e7-a144-e32d61c633cb None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:52.009454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a1994aa2-9aba-49e7-a144-e32d61c633cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:52.019726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1009/1997] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:52.020636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e79a-5cab-4b16-9e67-5e3af0255410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.021480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e79a-5cab-4b16-9e67-5e3af0255410 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.022917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e79a-5cab-4b16-9e67-5e3af0255410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.023326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19e79a-5cab-4b16-9e67-5e3af0255410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.024343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19e79a-5cab-4b16-9e67-5e3af0255410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.028634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a19e79a-5cab-4b16-9e67-5e3af0255410 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.029585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88e07f-fa63-46d0-a4c6-9341eb09f306 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.029874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88e07f-fa63-46d0-a4c6-9341eb09f306 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.030186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc88e07f-fa63-46d0-a4c6-9341eb09f306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.043734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 989/1998] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:52.046970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e79a-5cab-4b16-9e67-5e3af0255410 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.062469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19e79a-5cab-4b16-9e67-5e3af0255410 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.065459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-fc88e07f-fa63-46d0-a4c6-9341eb09f306 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:52.066754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fc88e07f-fa63-46d0-a4c6-9341eb09f306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:52.069271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c63bca-0c2c-4515-96ae-5a061f550451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.070363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c63bca-0c2c-4515-96ae-5a061f550451 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.070997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c63bca-0c2c-4515-96ae-5a061f550451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.071468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c63bca-0c2c-4515-96ae-5a061f550451 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.072367 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c63bca-0c2c-4515-96ae-5a061f550451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.074925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c63bca-0c2c-4515-96ae-5a061f550451 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.091101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1010/1999] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:52.092958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 990/2000] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:52.098882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88e07f-fa63-46d0-a4c6-9341eb09f306 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.108091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c63bca-0c2c-4515-96ae-5a061f550451 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.109535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc88e07f-fa63-46d0-a4c6-9341eb09f306 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.115328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e3cc8-10c5-4254-862f-3ea987524365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.116015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e3cc8-10c5-4254-862f-3ea987524365 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.116304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e3cc8-10c5-4254-862f-3ea987524365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.116581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e3cc8-10c5-4254-862f-3ea987524365 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.117322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e3cc8-10c5-4254-862f-3ea987524365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:52.117633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e3cc8-10c5-4254-862f-3ea987524365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.119298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e3cc8-10c5-4254-862f-3ea987524365 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.125251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c63bca-0c2c-4515-96ae-5a061f550451 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.132684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31a8fc-6b7c-48ad-ae45-06808ec6eb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.133904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a8fc-6b7c-48ad-ae45-06808ec6eb38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.134398 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a8fc-6b7c-48ad-ae45-06808ec6eb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.134866 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31a8fc-6b7c-48ad-ae45-06808ec6eb38 admin admin] PATH_INFO: `/v3/projects/8df1fd63cbe04210b54f37792779b1fe/users/f1fc80bd3eb041348e5da519766332d7/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.137940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1011/2001] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:05:52.138866 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e3cc8-10c5-4254-862f-3ea987524365 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.150474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e3cc8-10c5-4254-862f-3ea987524365 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.155967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b777d62-3888-4014-a065-574ed8fdf466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.156644 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b777d62-3888-4014-a065-574ed8fdf466 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.156959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b777d62-3888-4014-a065-574ed8fdf466 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.157197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b777d62-3888-4014-a065-574ed8fdf466 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.158181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b777d62-3888-4014-a065-574ed8fdf466 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.158319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31a8fc-6b7c-48ad-ae45-06808ec6eb38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df1fd63cbe04210b54f37792779b1fe, user_id=f1fc80bd3eb041348e5da519766332d7, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.160256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b777d62-3888-4014-a065-574ed8fdf466 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.161971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31a8fc-6b7c-48ad-ae45-06808ec6eb38 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.172867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:52.192822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 991/2002] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:52 2022] PUT /identity/v3/projects/8df1fd63cbe04210b54f37792779b1fe/users/f1fc80bd3eb041348e5da519766332d7/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:52.203943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead812b-71f7-4736-8b68-2ac8baae5f4c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.204244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead812b-71f7-4736-8b68-2ac8baae5f4c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.204528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead812b-71f7-4736-8b68-2ac8baae5f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.208914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1012/2003] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:52 2022] POST /identity/v3/projects => generated 379 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:52.211391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b777d62-3888-4014-a065-574ed8fdf466 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.245770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cead812b-71f7-4736-8b68-2ac8baae5f4c None None] MFA Rules not processed for user `f1fc80bd3eb041348e5da519766332d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:52.247111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cead812b-71f7-4736-8b68-2ac8baae5f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:52.326676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b777d62-3888-4014-a065-574ed8fdf466 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.332645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 992/2004] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:52 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:52.333687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead812b-71f7-4736-8b68-2ac8baae5f4c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.334581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be343a-0fe1-4879-85d4-21bcc43418bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.335814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be343a-0fe1-4879-85d4-21bcc43418bb admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.336178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be343a-0fe1-4879-85d4-21bcc43418bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.336536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1be343a-0fe1-4879-85d4-21bcc43418bb admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.337358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be343a-0fe1-4879-85d4-21bcc43418bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.341345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1be343a-0fe1-4879-85d4-21bcc43418bb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.348115 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead812b-71f7-4736-8b68-2ac8baae5f4c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.356223 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31e756-a9b1-496b-8ce4-da49257f3d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.357552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31e756-a9b1-496b-8ce4-da49257f3d09 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.357835 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31e756-a9b1-496b-8ce4-da49257f3d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.358250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f31e756-a9b1-496b-8ce4-da49257f3d09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.360439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31e756-a9b1-496b-8ce4-da49257f3d09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.362701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f31e756-a9b1-496b-8ce4-da49257f3d09 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.379672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1013/2005] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:52 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:52.380939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be343a-0fe1-4879-85d4-21bcc43418bb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.394596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1be343a-0fe1-4879-85d4-21bcc43418bb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.400167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855c474-75f9-4ca6-aa0a-6239e008c371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.401096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855c474-75f9-4ca6-aa0a-6239e008c371 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.401350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855c474-75f9-4ca6-aa0a-6239e008c371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.401633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 993/2006] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:52 2022] POST /identity/v3/projects => generated 365 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:52.401827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7855c474-75f9-4ca6-aa0a-6239e008c371 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.402786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855c474-75f9-4ca6-aa0a-6239e008c371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.404718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7855c474-75f9-4ca6-aa0a-6239e008c371 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.406827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31e756-a9b1-496b-8ce4-da49257f3d09 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.417136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1014/2007] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:52.422084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855c474-75f9-4ca6-aa0a-6239e008c371 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.433052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7855c474-75f9-4ca6-aa0a-6239e008c371 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.455009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:52.496095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21467cf-1b6f-4c8b-8308-0b8213104d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.497020 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21467cf-1b6f-4c8b-8308-0b8213104d35 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.497744 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21467cf-1b6f-4c8b-8308-0b8213104d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.497744 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21467cf-1b6f-4c8b-8308-0b8213104d35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.498626 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21467cf-1b6f-4c8b-8308-0b8213104d35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.500869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21467cf-1b6f-4c8b-8308-0b8213104d35 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.503745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f31e756-a9b1-496b-8ce4-da49257f3d09 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.510343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f528fee-c447-497e-b756-0aa43ce8e995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.511411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f528fee-c447-497e-b756-0aa43ce8e995 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.511699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f528fee-c447-497e-b756-0aa43ce8e995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.512074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f528fee-c447-497e-b756-0aa43ce8e995 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.512778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f528fee-c447-497e-b756-0aa43ce8e995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.515297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f528fee-c447-497e-b756-0aa43ce8e995 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.517684 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1015/2008] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:52.518788 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21467cf-1b6f-4c8b-8308-0b8213104d35 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.529948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21467cf-1b6f-4c8b-8308-0b8213104d35 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.536786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee036b7f-916b-4bf3-aa17-ad3365d2ef04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.538083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee036b7f-916b-4bf3-aa17-ad3365d2ef04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.538487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee036b7f-916b-4bf3-aa17-ad3365d2ef04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.538873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee036b7f-916b-4bf3-aa17-ad3365d2ef04 admin admin] PATH_INFO: `/v3/projects/c82f260fd3b3495d956d705c2c5fd8c7/users/cc6e22cd8e374c2dbe5edc9962516204/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.551331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 994/2009] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:52 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:52.552548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f528fee-c447-497e-b756-0aa43ce8e995 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.559065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee036b7f-916b-4bf3-aa17-ad3365d2ef04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82f260fd3b3495d956d705c2c5fd8c7, user_id=cc6e22cd8e374c2dbe5edc9962516204, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.561321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee036b7f-916b-4bf3-aa17-ad3365d2ef04 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.566103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f528fee-c447-497e-b756-0aa43ce8e995 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.573255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26142030-a743-478c-b7d4-5c4f1e531e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.574074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26142030-a743-478c-b7d4-5c4f1e531e50 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.574655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26142030-a743-478c-b7d4-5c4f1e531e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.575054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26142030-a743-478c-b7d4-5c4f1e531e50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.576054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26142030-a743-478c-b7d4-5c4f1e531e50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.578100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26142030-a743-478c-b7d4-5c4f1e531e50 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.594370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1016/2010] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:52 2022] PUT /identity/v3/projects/c82f260fd3b3495d956d705c2c5fd8c7/users/cc6e22cd8e374c2dbe5edc9962516204/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:52.595179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 995/2011] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:52.598587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b703f7d-d7b6-48b7-b1c6-4a16726ef056 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.598743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b703f7d-d7b6-48b7-b1c6-4a16726ef056 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.599577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b703f7d-d7b6-48b7-b1c6-4a16726ef056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.599776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26142030-a743-478c-b7d4-5c4f1e531e50 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.616142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26142030-a743-478c-b7d4-5c4f1e531e50 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.643291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6b703f7d-d7b6-48b7-b1c6-4a16726ef056 None None] MFA Rules not processed for user `cc6e22cd8e374c2dbe5edc9962516204`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:52.644777 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6b703f7d-d7b6-48b7-b1c6-4a16726ef056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:52.694058 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:52.719419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a51da-b4d4-44a3-bd69-b37f7795cddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.720671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a51da-b4d4-44a3-bd69-b37f7795cddd admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.720956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a51da-b4d4-44a3-bd69-b37f7795cddd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.721234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074a51da-b4d4-44a3-bd69-b37f7795cddd admin admin] PATH_INFO: `/v3/projects/ec774fbbfffd428888e4e6d872d3a47b/users/5ce8cc1a174d4746afaf3c18faf89207/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.745211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a51da-b4d4-44a3-bd69-b37f7795cddd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec774fbbfffd428888e4e6d872d3a47b, user_id=5ce8cc1a174d4746afaf3c18faf89207, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.748813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074a51da-b4d4-44a3-bd69-b37f7795cddd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.762702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1017/2012] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:52.767138 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b703f7d-d7b6-48b7-b1c6-4a16726ef056 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.779645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 996/2013] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:52 2022] PUT /identity/v3/projects/ec774fbbfffd428888e4e6d872d3a47b/users/5ce8cc1a174d4746afaf3c18faf89207/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:52.787748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a51da-b4d4-44a3-bd69-b37f7795cddd admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.804355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b703f7d-d7b6-48b7-b1c6-4a16726ef056 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.894738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b88cf2-30b6-4fc6-bac7-0434e1b6a437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:52.895439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b88cf2-30b6-4fc6-bac7-0434e1b6a437 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:52.895683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b88cf2-30b6-4fc6-bac7-0434e1b6a437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:52.895973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b88cf2-30b6-4fc6-bac7-0434e1b6a437 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:52.896656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b88cf2-30b6-4fc6-bac7-0434e1b6a437 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:52.899214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b88cf2-30b6-4fc6-bac7-0434e1b6a437 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:52.934454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1018/2014] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:52 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:52.938277 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b88cf2-30b6-4fc6-bac7-0434e1b6a437 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:52.940364 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074a51da-b4d4-44a3-bd69-b37f7795cddd admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:52.994281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:53.036031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b88cf2-30b6-4fc6-bac7-0434e1b6a437 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:53.042410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a20a5a-47d6-49ff-acdf-dcdb68b4cdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:53.042933 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a20a5a-47d6-49ff-acdf-dcdb68b4cdf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.043113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a20a5a-47d6-49ff-acdf-dcdb68b4cdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.043388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a20a5a-47d6-49ff-acdf-dcdb68b4cdf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.044170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a20a5a-47d6-49ff-acdf-dcdb68b4cdf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:53.046084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a20a5a-47d6-49ff-acdf-dcdb68b4cdf0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:53.075767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1019/2015] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:52 2022] POST /identity/v3/users => generated 297 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:05:53.076682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a20a5a-47d6-49ff-acdf-dcdb68b4cdf0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:53.088981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476e9bc-1969-4491-95fc-0c0537f1f733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:53.089672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476e9bc-1969-4491-95fc-0c0537f1f733 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.089959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476e9bc-1969-4491-95fc-0c0537f1f733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.090375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476e9bc-1969-4491-95fc-0c0537f1f733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.091147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476e9bc-1969-4491-95fc-0c0537f1f733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:53.092876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476e9bc-1969-4491-95fc-0c0537f1f733 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:53.105490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a20a5a-47d6-49ff-acdf-dcdb68b4cdf0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:53.113008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9fb4f-987a-4c5e-b091-c16419ef2569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:53.113695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9fb4f-987a-4c5e-b091-c16419ef2569 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.114030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9fb4f-987a-4c5e-b091-c16419ef2569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.114352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9fb4f-987a-4c5e-b091-c16419ef2569 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.115081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9fb4f-987a-4c5e-b091-c16419ef2569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:53.115613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 997/2016] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:53.118897 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9fb4f-987a-4c5e-b091-c16419ef2569 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:53.120833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476e9bc-1969-4491-95fc-0c0537f1f733 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:53.133332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1020/2017] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:53.134677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9fb4f-987a-4c5e-b091-c16419ef2569 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:53.136674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476e9bc-1969-4491-95fc-0c0537f1f733 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:53.143746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248fbed-854c-42e6-8985-54466717a619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:53.145322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9fb4f-987a-4c5e-b091-c16419ef2569 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:53.145588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248fbed-854c-42e6-8985-54466717a619 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.146305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248fbed-854c-42e6-8985-54466717a619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.146979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248fbed-854c-42e6-8985-54466717a619 admin admin] PATH_INFO: `/v3/projects/ec774fbbfffd428888e4e6d872d3a47b/users/5ce8cc1a174d4746afaf3c18faf89207/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.149994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248fbed-854c-42e6-8985-54466717a619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec774fbbfffd428888e4e6d872d3a47b, user_id=5ce8cc1a174d4746afaf3c18faf89207, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:53.150244 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f47f0-0b86-4943-be36-884f554cc159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:53.152776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f47f0-0b86-4943-be36-884f554cc159 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.152776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f47f0-0b86-4943-be36-884f554cc159 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.152776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f47f0-0b86-4943-be36-884f554cc159 admin admin] PATH_INFO: `/v3/projects/bfb1bb8d2f5d4b7f81077fe38fbd8c22/users/5fb94229ddd54ac1b8bce7a086d5b3e0/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.162461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248fbed-854c-42e6-8985-54466717a619 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:53.184166 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f47f0-0b86-4943-be36-884f554cc159 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb1bb8d2f5d4b7f81077fe38fbd8c22, user_id=5fb94229ddd54ac1b8bce7a086d5b3e0, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:53.194091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 998/2018] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:53 2022] PUT /identity/v3/projects/ec774fbbfffd428888e4e6d872d3a47b/users/5ce8cc1a174d4746afaf3c18faf89207/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:53.196999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699f47f0-0b86-4943-be36-884f554cc159 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:53.199184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbb5ab-1778-4e31-9a00-ca8ae11ad810 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.199740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbb5ab-1778-4e31-9a00-ca8ae11ad810 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.200025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bbb5ab-1778-4e31-9a00-ca8ae11ad810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.231858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-88bbb5ab-1778-4e31-9a00-ca8ae11ad810 None None] MFA Rules not processed for user `5ce8cc1a174d4746afaf3c18faf89207`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:53.233081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-88bbb5ab-1778-4e31-9a00-ca8ae11ad810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:53.235925 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1021/2019] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:53 2022] PUT /identity/v3/projects/bfb1bb8d2f5d4b7f81077fe38fbd8c22/users/5fb94229ddd54ac1b8bce7a086d5b3e0/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:53.240176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f47f0-0b86-4943-be36-884f554cc159 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:53.256483 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:53.327976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f47f0-0b86-4943-be36-884f554cc159 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:53.333716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1569a3-72ed-4412-bcca-23affab04c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:53.334720 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1569a3-72ed-4412-bcca-23affab04c99 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.335253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1569a3-72ed-4412-bcca-23affab04c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.335755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1569a3-72ed-4412-bcca-23affab04c99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.337404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1569a3-72ed-4412-bcca-23affab04c99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:53.340085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1569a3-72ed-4412-bcca-23affab04c99 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:53.344712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 999/2020] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:53.358617 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbb5ab-1778-4e31-9a00-ca8ae11ad810 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:53.361067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-88bbb5ab-1778-4e31-9a00-ca8ae11ad810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:53.364538 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1022/2021] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:53.368672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1569a3-72ed-4412-bcca-23affab04c99 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:53.385300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1569a3-72ed-4412-bcca-23affab04c99 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:53.397603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e81260-ad7a-47da-aba4-5d34941c0676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:53.399445 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e81260-ad7a-47da-aba4-5d34941c0676 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.400432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e81260-ad7a-47da-aba4-5d34941c0676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.400945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e81260-ad7a-47da-aba4-5d34941c0676 admin admin] PATH_INFO: `/v3/projects/bfb1bb8d2f5d4b7f81077fe38fbd8c22/users/5fb94229ddd54ac1b8bce7a086d5b3e0/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.404193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e81260-ad7a-47da-aba4-5d34941c0676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb1bb8d2f5d4b7f81077fe38fbd8c22, user_id=5fb94229ddd54ac1b8bce7a086d5b3e0, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:53.408373 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e81260-ad7a-47da-aba4-5d34941c0676 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:53.429140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1023/2022] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:53 2022] PUT /identity/v3/projects/bfb1bb8d2f5d4b7f81077fe38fbd8c22/users/5fb94229ddd54ac1b8bce7a086d5b3e0/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:53.440551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f701313-4033-421f-a4a7-68065173bc80 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.441352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f701313-4033-421f-a4a7-68065173bc80 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.441352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f701313-4033-421f-a4a7-68065173bc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.459582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bbb5ab-1778-4e31-9a00-ca8ae11ad810 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:53.473095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3f701313-4033-421f-a4a7-68065173bc80 None None] MFA Rules not processed for user `5fb94229ddd54ac1b8bce7a086d5b3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:53.474818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3f701313-4033-421f-a4a7-68065173bc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:53.561310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad278d5-1ab2-4b05-8b05-4f41871c3f19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:53.562639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad278d5-1ab2-4b05-8b05-4f41871c3f19 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.563108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad278d5-1ab2-4b05-8b05-4f41871c3f19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.563501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad278d5-1ab2-4b05-8b05-4f41871c3f19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.579202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:53.612806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1024/2023] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:05:53.614286 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4591a7-faa9-4dc2-a581-45c17295cd43 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.614583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4591a7-faa9-4dc2-a581-45c17295cd43 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.614867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4591a7-faa9-4dc2-a581-45c17295cd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.656845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad278d5-1ab2-4b05-8b05-4f41871c3f19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:53.657096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dc4591a7-faa9-4dc2-a581-45c17295cd43 None None] MFA Rules not processed for user `c31dbb3dcd9a411ea125573c8189e00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:53.658577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dc4591a7-faa9-4dc2-a581-45c17295cd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:53.660481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad278d5-1ab2-4b05-8b05-4f41871c3f19 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:53.674974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1000/2024] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:53.676047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad278d5-1ab2-4b05-8b05-4f41871c3f19 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:53.677765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fad278d5-1ab2-4b05-8b05-4f41871c3f19 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:53.692604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad278d5-1ab2-4b05-8b05-4f41871c3f19 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:53.696831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae780bd-b065-4184-93d4-a5ac2d392db3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:53.697609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae780bd-b065-4184-93d4-a5ac2d392db3 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.697998 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae780bd-b065-4184-93d4-a5ac2d392db3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.698486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae780bd-b065-4184-93d4-a5ac2d392db3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.710013 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae780bd-b065-4184-93d4-a5ac2d392db3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:53.711780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae780bd-b065-4184-93d4-a5ac2d392db3 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:53.723972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1001/2025] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:53.741637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1025/2026] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:53.759299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae780bd-b065-4184-93d4-a5ac2d392db3 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:53.779972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae780bd-b065-4184-93d4-a5ac2d392db3 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:53.803082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5db0b7-c1ca-49a5-a42d-0835030f50bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:53.803737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5db0b7-c1ca-49a5-a42d-0835030f50bf service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:53.804030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5db0b7-c1ca-49a5-a42d-0835030f50bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:53.804389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5db0b7-c1ca-49a5-a42d-0835030f50bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:53.844408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5db0b7-c1ca-49a5-a42d-0835030f50bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:53.845994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5db0b7-c1ca-49a5-a42d-0835030f50bf service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:53.885811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1002/2027] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:54.841857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4591a7-faa9-4dc2-a581-45c17295cd43 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:54.872950 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:55.039011 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4591a7-faa9-4dc2-a581-45c17295cd43 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:55.047396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:55.048960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:55.049341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:55.050607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] PATH_INFO: `/v3/users/cc6e22cd8e374c2dbe5edc9962516204` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:55.052249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc6e22cd8e374c2dbe5edc9962516204)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:55.055274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:55.137561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc6e22cd8e374c2dbe5edc9962516204'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:55.138557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc6e22cd8e374c2dbe5edc9962516204'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:55.154984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc6e22cd8e374c2dbe5edc9962516204'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:55.172892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc6e22cd8e374c2dbe5edc9962516204'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:55.173569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc6e22cd8e374c2dbe5edc9962516204'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:55.199735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1026/2028] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:54 2022] DELETE /identity/v3/users/cc6e22cd8e374c2dbe5edc9962516204 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:55.204630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116b43f-5ab9-457b-b657-b0e886cf3bc6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:55.205130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116b43f-5ab9-457b-b657-b0e886cf3bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:55.205419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116b43f-5ab9-457b-b657-b0e886cf3bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:55.233751 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6116b43f-5ab9-457b-b657-b0e886cf3bc6 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:55.235242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6116b43f-5ab9-457b-b657-b0e886cf3bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:55.326221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1003/2029] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:55.343540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:55.345655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:55.441709 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3c78b-6b7a-409e-9a5a-2cc4b0d1ff82 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:55.451261 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde2191-0557-4a27-9eb7-eadba860a94d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:55.452800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde2191-0557-4a27-9eb7-eadba860a94d service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:55.454225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde2191-0557-4a27-9eb7-eadba860a94d service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:55.457163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cde2191-0557-4a27-9eb7-eadba860a94d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:55.481286 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde2191-0557-4a27-9eb7-eadba860a94d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:55.486223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cde2191-0557-4a27-9eb7-eadba860a94d service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:55.517849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1027/2030] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:55.526490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b57b56-3d32-4f3e-8238-73f9b8cd7a33 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:55.526991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b57b56-3d32-4f3e-8238-73f9b8cd7a33 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:55.527374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b57b56-3d32-4f3e-8238-73f9b8cd7a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:55.569008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-33b57b56-3d32-4f3e-8238-73f9b8cd7a33 None None] MFA Rules not processed for user `c31dbb3dcd9a411ea125573c8189e00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:55.570435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-33b57b56-3d32-4f3e-8238-73f9b8cd7a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:55.677996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1004/2031] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:55 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:56.099825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde2191-0557-4a27-9eb7-eadba860a94d service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:56.101764 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2cde2191-0557-4a27-9eb7-eadba860a94d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:56.125474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cde2191-0557-4a27-9eb7-eadba860a94d service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:56.137383 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:56.138299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:56.138559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:56.138837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] PATH_INFO: `/v3/projects/c82f260fd3b3495d956d705c2c5fd8c7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:56.141734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82f260fd3b3495d956d705c2c5fd8c7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:56.143737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:56.185292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:56.228125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82f260fd3b3495d956d705c2c5fd8c7', 'request_id': 'req-94b85a77-eb30-4fef-b2be-b66af95e672c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.243544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82f260fd3b3495d956d705c2c5fd8c7', 'request_id': 'req-94b85a77-eb30-4fef-b2be-b66af95e672c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.256394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82f260fd3b3495d956d705c2c5fd8c7', 'request_id': 'req-94b85a77-eb30-4fef-b2be-b66af95e672c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.282858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] The token cache is being invalidate because project c82f260fd3b3495d956d705c2c5fd8c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:56.283159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.283898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.292571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1028/2032] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:56 2022] DELETE /identity/v3/projects/c82f260fd3b3495d956d705c2c5fd8c7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:56.293850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b57b56-3d32-4f3e-8238-73f9b8cd7a33 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:56.313940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b57b56-3d32-4f3e-8238-73f9b8cd7a33 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:56.392663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:56.427586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:56.428905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:56.429361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:56.430623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] PATH_INFO: `/v3/users/5fb94229ddd54ac1b8bce7a086d5b3e0` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:56.432138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb94229ddd54ac1b8bce7a086d5b3e0)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:56.434980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:56.535633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb94229ddd54ac1b8bce7a086d5b3e0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.538219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb94229ddd54ac1b8bce7a086d5b3e0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.538976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb94229ddd54ac1b8bce7a086d5b3e0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.557560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb94229ddd54ac1b8bce7a086d5b3e0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.571538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a9ebd320-6067-4cc5-81f4-43239fa1d680 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb94229ddd54ac1b8bce7a086d5b3e0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.595618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1005/2033] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:56 2022] DELETE /identity/v3/users/5fb94229ddd54ac1b8bce7a086d5b3e0 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:56.616553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:56.618081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:56.643657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b85a77-eb30-4fef-b2be-b66af95e672c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:56.751766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:56.752912 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:56.753762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:56.754383 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] PATH_INFO: `/v3/users/f1fc80bd3eb041348e5da519766332d7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:56.755878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1fc80bd3eb041348e5da519766332d7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:56.758032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:56.820385 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1fc80bd3eb041348e5da519766332d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.821110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1fc80bd3eb041348e5da519766332d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.831784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1fc80bd3eb041348e5da519766332d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.842000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1fc80bd3eb041348e5da519766332d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.842767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1fc80bd3eb041348e5da519766332d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:56.872647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1029/2034] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:56 2022] DELETE /identity/v3/users/f1fc80bd3eb041348e5da519766332d7 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:56.875076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d9d31-3af9-43d3-a86d-81846c4aaa76 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:56.875365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d9d31-3af9-43d3-a86d-81846c4aaa76 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:56.875664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9d9d31-3af9-43d3-a86d-81846c4aaa76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:56.909610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0d9d9d31-3af9-43d3-a86d-81846c4aaa76 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:56.911050 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0d9d9d31-3af9-43d3-a86d-81846c4aaa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:56.999020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1006/2035] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:57.010001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:57.011658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:57.057998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:57.110341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb66e362-54ff-43cf-9111-461d690074a4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:57.114483 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d0569-fddf-4487-ab43-e40da087ccbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:57.115233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d0569-fddf-4487-ab43-e40da087ccbe service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.115648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d0569-fddf-4487-ab43-e40da087ccbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.121583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d0569-fddf-4487-ab43-e40da087ccbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.130288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d9d31-3af9-43d3-a86d-81846c4aaa76 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:57.132285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0d9d9d31-3af9-43d3-a86d-81846c4aaa76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:57.145525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d0569-fddf-4487-ab43-e40da087ccbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:57.147183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d0569-fddf-4487-ab43-e40da087ccbe service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:57.161226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1030/2036] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:57.200565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9d9d31-3af9-43d3-a86d-81846c4aaa76 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:57.216374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:57.217260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.217586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.218382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] PATH_INFO: `/v3/projects/bfb1bb8d2f5d4b7f81077fe38fbd8c22` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.221980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb1bb8d2f5d4b7f81077fe38fbd8c22)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:57.225199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:57.320994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:57.325189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb1bb8d2f5d4b7f81077fe38fbd8c22', 'request_id': 'req-ea8cb96b-5f31-4302-995a-8f787e9e84dd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.344282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb1bb8d2f5d4b7f81077fe38fbd8c22', 'request_id': 'req-ea8cb96b-5f31-4302-995a-8f787e9e84dd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.351707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb1bb8d2f5d4b7f81077fe38fbd8c22', 'request_id': 'req-ea8cb96b-5f31-4302-995a-8f787e9e84dd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.377132 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] The token cache is being invalidate because project bfb1bb8d2f5d4b7f81077fe38fbd8c22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:57.377436 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.378788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ea8cb96b-5f31-4302-995a-8f787e9e84dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.381087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1007/2037] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:57 2022] DELETE /identity/v3/projects/bfb1bb8d2f5d4b7f81077fe38fbd8c22 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:57.388139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7048a1-96d4-49ef-96b0-d64e26ba7ac8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.388790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7048a1-96d4-49ef-96b0-d64e26ba7ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.389226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7048a1-96d4-49ef-96b0-d64e26ba7ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.438951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bc7048a1-96d4-49ef-96b0-d64e26ba7ac8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:57.443221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bc7048a1-96d4-49ef-96b0-d64e26ba7ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:57.576828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1031/2038] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:57.588118 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14cdbd-856c-4504-8f4d-dcf52688f9cd None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.588410 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14cdbd-856c-4504-8f4d-dcf52688f9cd None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.588689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14cdbd-856c-4504-8f4d-dcf52688f9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.613739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6d14cdbd-856c-4504-8f4d-dcf52688f9cd None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:57.614911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6d14cdbd-856c-4504-8f4d-dcf52688f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:57.641515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1008/2039] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:57.647620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7048a1-96d4-49ef-96b0-d64e26ba7ac8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:57.658595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14cdbd-856c-4504-8f4d-dcf52688f9cd None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:57.660990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6d14cdbd-856c-4504-8f4d-dcf52688f9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:57.667609 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7048a1-96d4-49ef-96b0-d64e26ba7ac8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:57.679922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14cdbd-856c-4504-8f4d-dcf52688f9cd None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:57.683429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c296f1-6976-49ae-8700-c042b995c977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:57.684455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c296f1-6976-49ae-8700-c042b995c977 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.684747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c296f1-6976-49ae-8700-c042b995c977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.685329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c296f1-6976-49ae-8700-c042b995c977 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.685707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:57.686112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c296f1-6976-49ae-8700-c042b995c977 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:57.686475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c296f1-6976-49ae-8700-c042b995c977 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:57.686945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.687324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.687699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] PATH_INFO: `/v3/projects/8df1fd63cbe04210b54f37792779b1fe` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.688457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c296f1-6976-49ae-8700-c042b995c977 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:57.689577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df1fd63cbe04210b54f37792779b1fe)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:57.691694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:57.710339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1032/2040] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:05:57.711315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c296f1-6976-49ae-8700-c042b995c977 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:57.722237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c296f1-6976-49ae-8700-c042b995c977 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:57.727107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d62a54-7290-4e6d-b414-064b9fc2e77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:57.727766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d62a54-7290-4e6d-b414-064b9fc2e77b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.728055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d62a54-7290-4e6d-b414-064b9fc2e77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.728338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d62a54-7290-4e6d-b414-064b9fc2e77b admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.729069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d62a54-7290-4e6d-b414-064b9fc2e77b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:57.731097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d62a54-7290-4e6d-b414-064b9fc2e77b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:57.762330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df1fd63cbe04210b54f37792779b1fe', 'request_id': 'req-8a342623-0d35-4770-98db-af37583831cf'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.768021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1033/2041] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:57 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:57.769579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d62a54-7290-4e6d-b414-064b9fc2e77b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:57.773146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df1fd63cbe04210b54f37792779b1fe', 'request_id': 'req-8a342623-0d35-4770-98db-af37583831cf'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.780624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df1fd63cbe04210b54f37792779b1fe', 'request_id': 'req-8a342623-0d35-4770-98db-af37583831cf'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.795118 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] The token cache is being invalidate because project 8df1fd63cbe04210b54f37792779b1fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:57.795408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.796212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.798072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1009/2042] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:57 2022] DELETE /identity/v3/projects/8df1fd63cbe04210b54f37792779b1fe => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:57.801659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:57.803075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:57.841201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d62a54-7290-4e6d-b414-064b9fc2e77b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:57.847006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d6da9-7a7d-4bd1-84fe-e75929e28f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:57.847684 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d6da9-7a7d-4bd1-84fe-e75929e28f29 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.847968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d6da9-7a7d-4bd1-84fe-e75929e28f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.848248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d6da9-7a7d-4bd1-84fe-e75929e28f29 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.848945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d6da9-7a7d-4bd1-84fe-e75929e28f29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:57.850902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d6da9-7a7d-4bd1-84fe-e75929e28f29 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:57.879229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1034/2043] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:57 2022] POST /identity/v3/users => generated 288 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:57.879801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a342623-0d35-4770-98db-af37583831cf admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:57.880428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d6da9-7a7d-4bd1-84fe-e75929e28f29 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:57.882127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1a6d6da9-7a7d-4bd1-84fe-e75929e28f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:57.885800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:57.886708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.886994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.887269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] PATH_INFO: `/v3/users/5ce8cc1a174d4746afaf3c18faf89207` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.888908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce8cc1a174d4746afaf3c18faf89207)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:57.892661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:57.896062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d6da9-7a7d-4bd1-84fe-e75929e28f29 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:57.901432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe0ad5-7235-4742-a51c-5387d9a93bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:57.902127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe0ad5-7235-4742-a51c-5387d9a93bec admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.902419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe0ad5-7235-4742-a51c-5387d9a93bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.902700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebe0ad5-7235-4742-a51c-5387d9a93bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.903432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe0ad5-7235-4742-a51c-5387d9a93bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:57.905020 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebe0ad5-7235-4742-a51c-5387d9a93bec admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:57.922345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1035/2044] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:57.924782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe0ad5-7235-4742-a51c-5387d9a93bec admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:57.938731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebe0ad5-7235-4742-a51c-5387d9a93bec admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:57.944248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c0549-02db-4776-891a-77f13da19302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:57.944921 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c0549-02db-4776-891a-77f13da19302 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.945208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c0549-02db-4776-891a-77f13da19302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.945499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670c0549-02db-4776-891a-77f13da19302 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.946527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c0549-02db-4776-891a-77f13da19302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:57.948225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670c0549-02db-4776-891a-77f13da19302 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:57.956098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce8cc1a174d4746afaf3c18faf89207'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.956779 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce8cc1a174d4746afaf3c18faf89207'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.957401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce8cc1a174d4746afaf3c18faf89207'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.960186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1036/2045] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:05:57.962080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:57.964448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c0549-02db-4776-891a-77f13da19302 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:57.966349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-670c0549-02db-4776-891a-77f13da19302 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:57.974948 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce8cc1a174d4746afaf3c18faf89207'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.981859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670c0549-02db-4776-891a-77f13da19302 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:57.985464 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-11e59989-bead-49ad-b476-58336d2a5f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce8cc1a174d4746afaf3c18faf89207'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:57.988146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508d2fe-d67d-47d9-8948-5ead8034e8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:57.989151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508d2fe-d67d-47d9-8948-5ead8034e8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:57.989352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508d2fe-d67d-47d9-8948-5ead8034e8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:57.989638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508d2fe-d67d-47d9-8948-5ead8034e8bd admin admin] PATH_INFO: `/v3/projects/3851eeeb702a4986bd91da32153fbb36/users/40e7ab96a0e34a1ab02e7243362ca215/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:57.992274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:57.999619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1010/2046] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:05:57 2022] DELETE /identity/v3/users/5ce8cc1a174d4746afaf3c18faf89207 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:58.008974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508d2fe-d67d-47d9-8948-5ead8034e8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3851eeeb702a4986bd91da32153fbb36, user_id=40e7ab96a0e34a1ab02e7243362ca215, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:58.011699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508d2fe-d67d-47d9-8948-5ead8034e8bd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:58.026034 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1037/2047] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:57 2022] PUT /identity/v3/projects/3851eeeb702a4986bd91da32153fbb36/users/40e7ab96a0e34a1ab02e7243362ca215/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:58.036449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e8ae-50c3-41f6-bc6c-074ca81ace34 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.036725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e8ae-50c3-41f6-bc6c-074ca81ace34 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.037144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7982e8ae-50c3-41f6-bc6c-074ca81ace34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.067677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7982e8ae-50c3-41f6-bc6c-074ca81ace34 None None] MFA Rules not processed for user `40e7ab96a0e34a1ab02e7243362ca215`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:58.069041 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7982e8ae-50c3-41f6-bc6c-074ca81ace34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:58.144240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1011/2048] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:58.154861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508d2fe-d67d-47d9-8948-5ead8034e8bd admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:58.156906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7508d2fe-d67d-47d9-8948-5ead8034e8bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:58.220253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508d2fe-d67d-47d9-8948-5ead8034e8bd admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:58.224021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121034bd-c081-4f3a-9da1-db778de74fcb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:58.224669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121034bd-c081-4f3a-9da1-db778de74fcb service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.224952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121034bd-c081-4f3a-9da1-db778de74fcb service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.225497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121034bd-c081-4f3a-9da1-db778de74fcb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.234339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121034bd-c081-4f3a-9da1-db778de74fcb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:58.235710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121034bd-c081-4f3a-9da1-db778de74fcb service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:58.245612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1038/2049] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:05:58.288595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982e8ae-50c3-41f6-bc6c-074ca81ace34 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:58.290127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7982e8ae-50c3-41f6-bc6c-074ca81ace34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:58.416264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7982e8ae-50c3-41f6-bc6c-074ca81ace34 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:58.416264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:58.416264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.416264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.417149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] PATH_INFO: `/v3/projects/ec774fbbfffd428888e4e6d872d3a47b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.417607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec774fbbfffd428888e4e6d872d3a47b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:58.420107 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:58.467383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec774fbbfffd428888e4e6d872d3a47b', 'request_id': 'req-266bacd9-ec56-4cd9-b75f-171c5d2864a1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:58.476076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec774fbbfffd428888e4e6d872d3a47b', 'request_id': 'req-266bacd9-ec56-4cd9-b75f-171c5d2864a1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:58.481069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:58.483634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec774fbbfffd428888e4e6d872d3a47b', 'request_id': 'req-266bacd9-ec56-4cd9-b75f-171c5d2864a1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:58.498057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] The token cache is being invalidate because project ec774fbbfffd428888e4e6d872d3a47b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:05:58.498330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:58.498980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-266bacd9-ec56-4cd9-b75f-171c5d2864a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:05:58.501086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1012/2050] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:05:58 2022] DELETE /identity/v3/projects/ec774fbbfffd428888e4e6d872d3a47b => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:58.506196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afccb5-d7bd-4d7a-898e-740e784c5dc6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.506561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afccb5-d7bd-4d7a-898e-740e784c5dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.506909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0afccb5-d7bd-4d7a-898e-740e784c5dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.527220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c0afccb5-d7bd-4d7a-898e-740e784c5dc6 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:58.528789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c0afccb5-d7bd-4d7a-898e-740e784c5dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:58.597465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1039/2051] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:58.605959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6397e9-ed9b-47ba-9ac7-fd9f0055730a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.605959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6397e9-ed9b-47ba-9ac7-fd9f0055730a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.606299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6397e9-ed9b-47ba-9ac7-fd9f0055730a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.625101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dd6397e9-ed9b-47ba-9ac7-fd9f0055730a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:58.626417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dd6397e9-ed9b-47ba-9ac7-fd9f0055730a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:58.636252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1013/2052] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:58.640956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afccb5-d7bd-4d7a-898e-740e784c5dc6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:58.660076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0afccb5-d7bd-4d7a-898e-740e784c5dc6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:58.665031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f459c2-4c3b-4d82-a64f-a9a0e0089e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:58.665930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f459c2-4c3b-4d82-a64f-a9a0e0089e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.666351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f459c2-4c3b-4d82-a64f-a9a0e0089e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.666756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f459c2-4c3b-4d82-a64f-a9a0e0089e70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.668183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f459c2-4c3b-4d82-a64f-a9a0e0089e70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:05:58.668512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f459c2-4c3b-4d82-a64f-a9a0e0089e70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:58.670418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f459c2-4c3b-4d82-a64f-a9a0e0089e70 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:58.686097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1040/2053] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:05:58.691081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6397e9-ed9b-47ba-9ac7-fd9f0055730a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:58.701231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6397e9-ed9b-47ba-9ac7-fd9f0055730a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:58.706925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da1cf1-bc34-436a-bb6e-cff0f1d10fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:58.707700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1cf1-bc34-436a-bb6e-cff0f1d10fba admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.708075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1cf1-bc34-436a-bb6e-cff0f1d10fba admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.708460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da1cf1-bc34-436a-bb6e-cff0f1d10fba admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.709222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da1cf1-bc34-436a-bb6e-cff0f1d10fba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:58.711235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0da1cf1-bc34-436a-bb6e-cff0f1d10fba admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:58.746074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1014/2054] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:58 2022] POST /identity/v3/projects => generated 379 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:58.748191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f459c2-4c3b-4d82-a64f-a9a0e0089e70 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:58.759061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f459c2-4c3b-4d82-a64f-a9a0e0089e70 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:58.816233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57891c09-7437-44af-861e-5fe005f72902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:58.816914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57891c09-7437-44af-861e-5fe005f72902 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.817198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57891c09-7437-44af-861e-5fe005f72902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.817477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57891c09-7437-44af-861e-5fe005f72902 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.818210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57891c09-7437-44af-861e-5fe005f72902 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:58.820130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57891c09-7437-44af-861e-5fe005f72902 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:58.846212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1041/2055] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:58 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:58.849449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da1cf1-bc34-436a-bb6e-cff0f1d10fba admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:58.867374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da1cf1-bc34-436a-bb6e-cff0f1d10fba admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:58.872164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96caae-3d5b-4a34-8a2b-4a99fe44366b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:58.872831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96caae-3d5b-4a34-8a2b-4a99fe44366b admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.873151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96caae-3d5b-4a34-8a2b-4a99fe44366b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.873431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb96caae-3d5b-4a34-8a2b-4a99fe44366b admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.874773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96caae-3d5b-4a34-8a2b-4a99fe44366b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:58.876377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb96caae-3d5b-4a34-8a2b-4a99fe44366b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:58.888006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1015/2056] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:58.891856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57891c09-7437-44af-861e-5fe005f72902 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:58.897806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:58.904751 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57891c09-7437-44af-861e-5fe005f72902 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:58.913786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b68c9a-cbd2-470c-8cef-9e8b279090c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:58.914430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68c9a-cbd2-470c-8cef-9e8b279090c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.914717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68c9a-cbd2-470c-8cef-9e8b279090c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.914998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b68c9a-cbd2-470c-8cef-9e8b279090c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.915731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b68c9a-cbd2-470c-8cef-9e8b279090c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:58.917338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b68c9a-cbd2-470c-8cef-9e8b279090c4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:58.928557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1042/2057] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:58.932991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96caae-3d5b-4a34-8a2b-4a99fe44366b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:58.943808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb96caae-3d5b-4a34-8a2b-4a99fe44366b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:58.948624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fbcdb-94c4-463b-a3c6-b4e460dcb0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:58.949514 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fbcdb-94c4-463b-a3c6-b4e460dcb0de admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.949800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fbcdb-94c4-463b-a3c6-b4e460dcb0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.950110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22fbcdb-94c4-463b-a3c6-b4e460dcb0de admin admin] PATH_INFO: `/v3/projects/997a1d7b3630439ba2739232cc29b3e4/users/83304862f6744ceab7a8d71be0f2877d/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:58.965273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fbcdb-94c4-463b-a3c6-b4e460dcb0de admin admin] RBAC: Authorizing `identity:create_grant(project_id=997a1d7b3630439ba2739232cc29b3e4, user_id=83304862f6744ceab7a8d71be0f2877d, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:58.968159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22fbcdb-94c4-463b-a3c6-b4e460dcb0de admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:58.985803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1016/2058] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:58 2022] PUT /identity/v3/projects/997a1d7b3630439ba2739232cc29b3e4/users/83304862f6744ceab7a8d71be0f2877d/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:58.996569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b21d5f-81b5-4b41-ab0b-67faa4ae3bfc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:58.996862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b21d5f-81b5-4b41-ab0b-67faa4ae3bfc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:58.997145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b21d5f-81b5-4b41-ab0b-67faa4ae3bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:59.025386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-78b21d5f-81b5-4b41-ab0b-67faa4ae3bfc None None] MFA Rules not processed for user `83304862f6744ceab7a8d71be0f2877d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:59.026649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-78b21d5f-81b5-4b41-ab0b-67faa4ae3bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:59.097245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1043/2059] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:58 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:59.101630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fbcdb-94c4-463b-a3c6-b4e460dcb0de admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:59.166031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22fbcdb-94c4-463b-a3c6-b4e460dcb0de admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:59.171653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928ff60-0963-458c-ac88-3d0be9d0507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:59.172341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ff60-0963-458c-ac88-3d0be9d0507a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:59.172634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ff60-0963-458c-ac88-3d0be9d0507a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:59.172914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928ff60-0963-458c-ac88-3d0be9d0507a admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:59.173645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928ff60-0963-458c-ac88-3d0be9d0507a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:59.175637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928ff60-0963-458c-ac88-3d0be9d0507a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:59.204512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1017/2060] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:05:59 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:59.208022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b21d5f-81b5-4b41-ab0b-67faa4ae3bfc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:59.219850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b21d5f-81b5-4b41-ab0b-67faa4ae3bfc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:59.280470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b77292-4d76-4a6a-995f-1c5bd200d3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:59.281160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b77292-4d76-4a6a-995f-1c5bd200d3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:59.281447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b77292-4d76-4a6a-995f-1c5bd200d3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:59.281726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b77292-4d76-4a6a-995f-1c5bd200d3c8 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:59.282469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b77292-4d76-4a6a-995f-1c5bd200d3c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:59.284391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b77292-4d76-4a6a-995f-1c5bd200d3c8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:59.306073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1044/2061] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:05:59 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:05:59.309176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928ff60-0963-458c-ac88-3d0be9d0507a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:59.320262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928ff60-0963-458c-ac88-3d0be9d0507a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:59.325224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131df335-1969-4873-b4ed-cdacd525677e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:59.325897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df335-1969-4873-b4ed-cdacd525677e admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:59.326232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df335-1969-4873-b4ed-cdacd525677e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:59.327290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131df335-1969-4873-b4ed-cdacd525677e admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:59.328553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131df335-1969-4873-b4ed-cdacd525677e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:59.330933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131df335-1969-4873-b4ed-cdacd525677e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:59.343139 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1018/2062] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:59.346794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b77292-4d76-4a6a-995f-1c5bd200d3c8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:59.357208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b77292-4d76-4a6a-995f-1c5bd200d3c8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:59.366829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb92ce-cda8-4440-864f-7f26a6eec830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:59.366829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb92ce-cda8-4440-864f-7f26a6eec830 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:59.366829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb92ce-cda8-4440-864f-7f26a6eec830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:59.366829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb92ce-cda8-4440-864f-7f26a6eec830 admin admin] PATH_INFO: `/v3/projects/8e0ed575fce74493b2ef9a71fbeaf328/users/fdda843d263248b8ab58ae36c8e7e3d9/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:59.381806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb92ce-cda8-4440-864f-7f26a6eec830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0ed575fce74493b2ef9a71fbeaf328, user_id=fdda843d263248b8ab58ae36c8e7e3d9, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:59.384706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb92ce-cda8-4440-864f-7f26a6eec830 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:59.399168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1045/2063] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:59 2022] PUT /identity/v3/projects/8e0ed575fce74493b2ef9a71fbeaf328/users/fdda843d263248b8ab58ae36c8e7e3d9/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:05:59.403152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131df335-1969-4873-b4ed-cdacd525677e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:59.415822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131df335-1969-4873-b4ed-cdacd525677e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:59.483650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc4ac2-90ee-49a2-99cc-30ce402625f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:59.484388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4ac2-90ee-49a2-99cc-30ce402625f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:59.484679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4ac2-90ee-49a2-99cc-30ce402625f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:59.484984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc4ac2-90ee-49a2-99cc-30ce402625f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:59.485880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc4ac2-90ee-49a2-99cc-30ce402625f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:59.487665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc4ac2-90ee-49a2-99cc-30ce402625f3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:59.489060 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:59.503731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1019/2064] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:05:59.504653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb92ce-cda8-4440-864f-7f26a6eec830 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:59.515892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb92ce-cda8-4440-864f-7f26a6eec830 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:59.521536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260289a-860c-43d1-9c1c-0b4ae7cd8166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:59.522831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260289a-860c-43d1-9c1c-0b4ae7cd8166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:59.523121 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260289a-860c-43d1-9c1c-0b4ae7cd8166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:59.523408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1260289a-860c-43d1-9c1c-0b4ae7cd8166 admin admin] PATH_INFO: `/v3/projects/8e0ed575fce74493b2ef9a71fbeaf328/users/fdda843d263248b8ab58ae36c8e7e3d9/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:59.525493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260289a-860c-43d1-9c1c-0b4ae7cd8166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e0ed575fce74493b2ef9a71fbeaf328, user_id=fdda843d263248b8ab58ae36c8e7e3d9, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:59.528785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1260289a-860c-43d1-9c1c-0b4ae7cd8166 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:59.540762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1046/2065] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:05:59 2022] PUT /identity/v3/projects/8e0ed575fce74493b2ef9a71fbeaf328/users/fdda843d263248b8ab58ae36c8e7e3d9/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:05:59.552203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8880496-78d0-4514-9930-993dbd6a2004 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:59.552501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8880496-78d0-4514-9930-993dbd6a2004 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:59.552784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8880496-78d0-4514-9930-993dbd6a2004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:59.580666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c8880496-78d0-4514-9930-993dbd6a2004 None None] MFA Rules not processed for user `fdda843d263248b8ab58ae36c8e7e3d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:05:59.581901 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c8880496-78d0-4514-9930-993dbd6a2004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:59.684067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1020/2066] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:05:59 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:05:59.700085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260289a-860c-43d1-9c1c-0b4ae7cd8166 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:05:59.701841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1260289a-860c-43d1-9c1c-0b4ae7cd8166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:05:59.741553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:05:59.774915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1260289a-860c-43d1-9c1c-0b4ae7cd8166 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:05:59.778852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc378ff-0ed1-4eaa-83b8-dbcc750998e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:05:59.779594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc378ff-0ed1-4eaa-83b8-dbcc750998e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:05:59.780637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc378ff-0ed1-4eaa-83b8-dbcc750998e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:05:59.780637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc378ff-0ed1-4eaa-83b8-dbcc750998e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:05:59.792501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc378ff-0ed1-4eaa-83b8-dbcc750998e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:05:59.794018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc378ff-0ed1-4eaa-83b8-dbcc750998e9 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:05:59.807920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1047/2067] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:05:59 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:00.163454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8880496-78d0-4514-9930-993dbd6a2004 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:00.173795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8880496-78d0-4514-9930-993dbd6a2004 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:00.178594 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6a041-efc7-49f3-91e9-fde0909428f2 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdda843d263248b8ab58ae36c8e7e3d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e0ed575fce74493b2ef9a71fbeaf328', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:00.179339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6a041-efc7-49f3-91e9-fde0909428f2 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:00.179718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6a041-efc7-49f3-91e9-fde0909428f2 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:00.180016 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6a041-efc7-49f3-91e9-fde0909428f2 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:00.180768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6a041-efc7-49f3-91e9-fde0909428f2 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:00.181086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6a041-efc7-49f3-91e9-fde0909428f2 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:00.182739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6a041-efc7-49f3-91e9-fde0909428f2 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:00.198535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1021/2068] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:06:00.199679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc378ff-0ed1-4eaa-83b8-dbcc750998e9 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:00.209097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc378ff-0ed1-4eaa-83b8-dbcc750998e9 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:00.213785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22d002-619a-42c3-b942-c9aed7ab41f1 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdda843d263248b8ab58ae36c8e7e3d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e0ed575fce74493b2ef9a71fbeaf328', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:00.214488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22d002-619a-42c3-b942-c9aed7ab41f1 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:00.214781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22d002-619a-42c3-b942-c9aed7ab41f1 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:00.215061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22d002-619a-42c3-b942-c9aed7ab41f1 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:00.215765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22d002-619a-42c3-b942-c9aed7ab41f1 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:00.217692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22d002-619a-42c3-b942-c9aed7ab41f1 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:00.245408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1048/2069] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:00 2022] POST /identity/v3/projects => generated 354 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:00.249561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6a041-efc7-49f3-91e9-fde0909428f2 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:00.260171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6a041-efc7-49f3-91e9-fde0909428f2 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:00.322337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9462b2-1b55-446e-9bce-6c55811e64b8 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdda843d263248b8ab58ae36c8e7e3d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e0ed575fce74493b2ef9a71fbeaf328', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:00.323046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9462b2-1b55-446e-9bce-6c55811e64b8 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:00.323335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9462b2-1b55-446e-9bce-6c55811e64b8 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:00.323619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9462b2-1b55-446e-9bce-6c55811e64b8 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:00.324354 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9462b2-1b55-446e-9bce-6c55811e64b8 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:00.324674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9462b2-1b55-446e-9bce-6c55811e64b8 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:00.326310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9462b2-1b55-446e-9bce-6c55811e64b8 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:00.342382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1022/2070] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:06:00.379643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22d002-619a-42c3-b942-c9aed7ab41f1 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:00.392137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22d002-619a-42c3-b942-c9aed7ab41f1 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:00.398073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fdda843d263248b8ab58ae36c8e7e3d9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8e0ed575fce74493b2ef9a71fbeaf328', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:00.398960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:00.399250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:00.399526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] PATH_INFO: `/v3/projects/1416ef07d11440e394975d6c3eefae1d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:00.401155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1416ef07d11440e394975d6c3eefae1d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:00.403112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:00.455184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1416ef07d11440e394975d6c3eefae1d', 'request_id': 'req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:00.462672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1416ef07d11440e394975d6c3eefae1d', 'request_id': 'req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:00.472087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1416ef07d11440e394975d6c3eefae1d', 'request_id': 'req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:00.487688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] The token cache is being invalidate because project 1416ef07d11440e394975d6c3eefae1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:00.488238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:00.488911 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9b2b65be-9452-4a2c-96fc-2ae177bb8bf6 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:00.491474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1049/2071] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:00 2022] DELETE /identity/v3/projects/1416ef07d11440e394975d6c3eefae1d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:00.676847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9462b2-1b55-446e-9bce-6c55811e64b8 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:00.679824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7a9462b2-1b55-446e-9bce-6c55811e64b8 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:00.744758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9462b2-1b55-446e-9bce-6c55811e64b8 tempest-VolumeQuotaClassesTest-1202374267 tempest-VolumeQuotaClassesTest-1202374267-project-admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:00.750463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:00.804114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:00.805240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:00.805523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:00.805827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] PATH_INFO: `/v3/users/83304862f6744ceab7a8d71be0f2877d` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:00.807062 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=83304862f6744ceab7a8d71be0f2877d)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:00.809239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:00.859043 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83304862f6744ceab7a8d71be0f2877d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:00.859706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83304862f6744ceab7a8d71be0f2877d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:00.860323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83304862f6744ceab7a8d71be0f2877d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:00.871896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83304862f6744ceab7a8d71be0f2877d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:00.880638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83304862f6744ceab7a8d71be0f2877d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:00.893765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1023/2072] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:00 2022] DELETE /identity/v3/users/83304862f6744ceab7a8d71be0f2877d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:00.898199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4cd67-b502-4dfa-b13d-5bc93cdc139d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:00.898639 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4cd67-b502-4dfa-b13d-5bc93cdc139d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:00.898997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa4cd67-b502-4dfa-b13d-5bc93cdc139d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:00.915794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dfa4cd67-b502-4dfa-b13d-5bc93cdc139d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:00.916997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dfa4cd67-b502-4dfa-b13d-5bc93cdc139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:00.974848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1050/2073] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:00.984295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:00.985689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:01.043085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df40ec-ee86-4140-b77f-f9c9487565bb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:01.046869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abb8dd-7dac-41a3-b44d-302f45c169c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:01.047526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abb8dd-7dac-41a3-b44d-302f45c169c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:01.047809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abb8dd-7dac-41a3-b44d-302f45c169c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:01.048113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42abb8dd-7dac-41a3-b44d-302f45c169c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:01.057523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abb8dd-7dac-41a3-b44d-302f45c169c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:01.059045 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42abb8dd-7dac-41a3-b44d-302f45c169c2 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:01.071137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1024/2074] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:01.411259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4cd67-b502-4dfa-b13d-5bc93cdc139d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:01.412674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dfa4cd67-b502-4dfa-b13d-5bc93cdc139d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:01.430962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa4cd67-b502-4dfa-b13d-5bc93cdc139d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:01.437727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:01.438654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:01.438914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:01.439236 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] PATH_INFO: `/v3/projects/997a1d7b3630439ba2739232cc29b3e4` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:01.440967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] RBAC: Authorizing `identity:delete_project(project_id=997a1d7b3630439ba2739232cc29b3e4)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:01.443142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:01.479592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:01.494800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '997a1d7b3630439ba2739232cc29b3e4', 'request_id': 'req-912d8033-8a99-4121-84f9-25a7bae8048a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:01.501008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '997a1d7b3630439ba2739232cc29b3e4', 'request_id': 'req-912d8033-8a99-4121-84f9-25a7bae8048a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:01.508371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '997a1d7b3630439ba2739232cc29b3e4', 'request_id': 'req-912d8033-8a99-4121-84f9-25a7bae8048a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:01.519227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] The token cache is being invalidate because project 997a1d7b3630439ba2739232cc29b3e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:01.519520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:01.520158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:01.521702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1051/2075] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:01 2022] DELETE /identity/v3/projects/997a1d7b3630439ba2739232cc29b3e4 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:01.525571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abb8dd-7dac-41a3-b44d-302f45c169c2 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:01.535610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42abb8dd-7dac-41a3-b44d-302f45c169c2 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:01.564153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:01.595539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:01.596404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:01.596706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:01.596975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] PATH_INFO: `/v3/users/fdda843d263248b8ab58ae36c8e7e3d9` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:01.598222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdda843d263248b8ab58ae36c8e7e3d9)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:01.600136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:01.648348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdda843d263248b8ab58ae36c8e7e3d9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:01.648990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdda843d263248b8ab58ae36c8e7e3d9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:01.649598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdda843d263248b8ab58ae36c8e7e3d9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:01.662022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdda843d263248b8ab58ae36c8e7e3d9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:01.669356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-238bea13-a95a-4772-ab82-0751cc06f8d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdda843d263248b8ab58ae36c8e7e3d9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:01.682288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1025/2076] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:01 2022] DELETE /identity/v3/users/fdda843d263248b8ab58ae36c8e7e3d9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:02.047324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.048749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:02.065127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d8033-8a99-4121-84f9-25a7bae8048a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.126064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.126948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.127233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.127509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] PATH_INFO: `/v3/projects/8e0ed575fce74493b2ef9a71fbeaf328` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.128953 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e0ed575fce74493b2ef9a71fbeaf328)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:02.130890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:02.175463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e0ed575fce74493b2ef9a71fbeaf328', 'request_id': 'req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:02.181245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e0ed575fce74493b2ef9a71fbeaf328', 'request_id': 'req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:02.189274 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e0ed575fce74493b2ef9a71fbeaf328', 'request_id': 'req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:02.200268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] The token cache is being invalidate because project 8e0ed575fce74493b2ef9a71fbeaf328 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:02.200556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:02.201175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-3ab4e87d-d46d-4c31-bd1d-5ff9e7eba0de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:02.202749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1052/2077] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:02 2022] DELETE /identity/v3/projects/8e0ed575fce74493b2ef9a71fbeaf328 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:02.207537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa7e8b-9624-4d39-aae7-da8583af920c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.207825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa7e8b-9624-4d39-aae7-da8583af920c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.208098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fa7e8b-9624-4d39-aae7-da8583af920c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.224881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-36fa7e8b-9624-4d39-aae7-da8583af920c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:02.226095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-36fa7e8b-9624-4d39-aae7-da8583af920c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:02.287087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1026/2078] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:02.294818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49934bdf-cc69-4de2-ac20-54ac7223135a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.295218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49934bdf-cc69-4de2-ac20-54ac7223135a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.295561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49934bdf-cc69-4de2-ac20-54ac7223135a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.314252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-49934bdf-cc69-4de2-ac20-54ac7223135a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:02.315567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-49934bdf-cc69-4de2-ac20-54ac7223135a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:02.323906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1053/2079] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:02.328091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa7e8b-9624-4d39-aae7-da8583af920c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.337751 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fa7e8b-9624-4d39-aae7-da8583af920c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.342276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa4802-65a0-4ae9-ba94-e1438696d21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.343457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4802-65a0-4ae9-ba94-e1438696d21a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.343736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4802-65a0-4ae9-ba94-e1438696d21a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.344067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa4802-65a0-4ae9-ba94-e1438696d21a admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.344792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa4802-65a0-4ae9-ba94-e1438696d21a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:02.345099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa4802-65a0-4ae9-ba94-e1438696d21a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:02.346734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa4802-65a0-4ae9-ba94-e1438696d21a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:02.359563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1027/2080] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:06:02.363988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49934bdf-cc69-4de2-ac20-54ac7223135a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.373285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49934bdf-cc69-4de2-ac20-54ac7223135a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.378306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6ed20-7314-4713-905b-cb4563a3b4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.378974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6ed20-7314-4713-905b-cb4563a3b4c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.379258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6ed20-7314-4713-905b-cb4563a3b4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.379531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6ed20-7314-4713-905b-cb4563a3b4c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.380230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6ed20-7314-4713-905b-cb4563a3b4c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:02.382150 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6ed20-7314-4713-905b-cb4563a3b4c8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:02.412920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1054/2081] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:02 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:02.416989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa4802-65a0-4ae9-ba94-e1438696d21a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.427512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa4802-65a0-4ae9-ba94-e1438696d21a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.459933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:02.485754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac92c3-085f-47ab-818a-b875e7a6e3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.486461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac92c3-085f-47ab-818a-b875e7a6e3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.486752 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac92c3-085f-47ab-818a-b875e7a6e3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.487033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ac92c3-085f-47ab-818a-b875e7a6e3b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.487739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac92c3-085f-47ab-818a-b875e7a6e3b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:02.489779 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ac92c3-085f-47ab-818a-b875e7a6e3b9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:02.510724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1028/2082] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:02 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:02.514472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6ed20-7314-4713-905b-cb4563a3b4c8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.524543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6ed20-7314-4713-905b-cb4563a3b4c8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.529544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfbead-999d-428b-9644-5e1137cf03d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.530844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfbead-999d-428b-9644-5e1137cf03d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.531137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfbead-999d-428b-9644-5e1137cf03d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.531525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfbead-999d-428b-9644-5e1137cf03d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.532345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfbead-999d-428b-9644-5e1137cf03d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:02.534242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfbead-999d-428b-9644-5e1137cf03d5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:02.545445 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1055/2083] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:02.549838 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac92c3-085f-47ab-818a-b875e7a6e3b9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.559714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ac92c3-085f-47ab-818a-b875e7a6e3b9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.564229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f97f65c-c96f-471b-b1f5-74183820a7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.564874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97f65c-c96f-471b-b1f5-74183820a7ce admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.565154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97f65c-c96f-471b-b1f5-74183820a7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.565426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f97f65c-c96f-471b-b1f5-74183820a7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.566174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97f65c-c96f-471b-b1f5-74183820a7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:02.567755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f97f65c-c96f-471b-b1f5-74183820a7ce admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:02.578483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1029/2084] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:02.582241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfbead-999d-428b-9644-5e1137cf03d5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.592555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfbead-999d-428b-9644-5e1137cf03d5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.597896 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a753de-b937-40d6-bcac-f473d36e594e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.598791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a753de-b937-40d6-bcac-f473d36e594e admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.599072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a753de-b937-40d6-bcac-f473d36e594e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.599342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a753de-b937-40d6-bcac-f473d36e594e admin admin] PATH_INFO: `/v3/projects/2a084cbaeffb4e44ba085664cc287943/users/e8ffe1dee7e34bb7bcdfe01723f8267c/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.607216 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:02.616605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a753de-b937-40d6-bcac-f473d36e594e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a084cbaeffb4e44ba085664cc287943, user_id=e8ffe1dee7e34bb7bcdfe01723f8267c, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:02.619746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a753de-b937-40d6-bcac-f473d36e594e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:02.634722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1056/2085] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:02 2022] PUT /identity/v3/projects/2a084cbaeffb4e44ba085664cc287943/users/e8ffe1dee7e34bb7bcdfe01723f8267c/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:02.645776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd1326-48e6-46de-8b07-9146b7c377bc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.646095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd1326-48e6-46de-8b07-9146b7c377bc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.646377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd1326-48e6-46de-8b07-9146b7c377bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.672855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-49cd1326-48e6-46de-8b07-9146b7c377bc None None] MFA Rules not processed for user `e8ffe1dee7e34bb7bcdfe01723f8267c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:02.674063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-49cd1326-48e6-46de-8b07-9146b7c377bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:02.735621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1030/2086] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:02.740060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a753de-b937-40d6-bcac-f473d36e594e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.799617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a753de-b937-40d6-bcac-f473d36e594e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.804201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735ffa4-b796-47cb-8b5d-925963107ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.804841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735ffa4-b796-47cb-8b5d-925963107ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.805120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735ffa4-b796-47cb-8b5d-925963107ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.805394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8735ffa4-b796-47cb-8b5d-925963107ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.806100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735ffa4-b796-47cb-8b5d-925963107ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:02.807970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8735ffa4-b796-47cb-8b5d-925963107ed0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:02.832404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1057/2087] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:02 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:02.836182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd1326-48e6-46de-8b07-9146b7c377bc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.847324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd1326-48e6-46de-8b07-9146b7c377bc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.904095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d057fe4-eed0-4607-b648-59e81bccfcb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.904759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057fe4-eed0-4607-b648-59e81bccfcb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.905044 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057fe4-eed0-4607-b648-59e81bccfcb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.905963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d057fe4-eed0-4607-b648-59e81bccfcb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.906677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d057fe4-eed0-4607-b648-59e81bccfcb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:02.908570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d057fe4-eed0-4607-b648-59e81bccfcb9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:02.928069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1031/2088] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:02 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:02.931987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735ffa4-b796-47cb-8b5d-925963107ed0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.942208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8735ffa4-b796-47cb-8b5d-925963107ed0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.947108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eb082-4834-429e-8f8d-9ab4a1fd8486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.947838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb082-4834-429e-8f8d-9ab4a1fd8486 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.948128 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb082-4834-429e-8f8d-9ab4a1fd8486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.948404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581eb082-4834-429e-8f8d-9ab4a1fd8486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.949125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eb082-4834-429e-8f8d-9ab4a1fd8486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:02.950772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581eb082-4834-429e-8f8d-9ab4a1fd8486 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:02.965210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1058/2089] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:02.966340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d057fe4-eed0-4607-b648-59e81bccfcb9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:02.977088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d057fe4-eed0-4607-b648-59e81bccfcb9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:02.981711 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d92d5-4f82-4422-a364-f70a1a4a62e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:02.982604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d92d5-4f82-4422-a364-f70a1a4a62e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:02.982889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d92d5-4f82-4422-a364-f70a1a4a62e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:02.983196 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d92d5-4f82-4422-a364-f70a1a4a62e8 admin admin] PATH_INFO: `/v3/projects/9708db9615c94fcbb9a76ac33e240628/users/f2fc7b5c60dd41118d336d26eb32a276/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:02.997658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d92d5-4f82-4422-a364-f70a1a4a62e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9708db9615c94fcbb9a76ac33e240628, user_id=f2fc7b5c60dd41118d336d26eb32a276, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:03.000871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d92d5-4f82-4422-a364-f70a1a4a62e8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:03.013871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1032/2090] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:02 2022] PUT /identity/v3/projects/9708db9615c94fcbb9a76ac33e240628/users/f2fc7b5c60dd41118d336d26eb32a276/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:03.018325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eb082-4834-429e-8f8d-9ab4a1fd8486 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:03.028038 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581eb082-4834-429e-8f8d-9ab4a1fd8486 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:03.084736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd291f2-a20e-43e8-a0d1-684768b52514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:03.085627 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd291f2-a20e-43e8-a0d1-684768b52514 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:03.085909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd291f2-a20e-43e8-a0d1-684768b52514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:03.086229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd291f2-a20e-43e8-a0d1-684768b52514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:03.086959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd291f2-a20e-43e8-a0d1-684768b52514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:03.088523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd291f2-a20e-43e8-a0d1-684768b52514 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:03.098704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1059/2091] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:03.102791 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d92d5-4f82-4422-a364-f70a1a4a62e8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:03.112996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d92d5-4f82-4422-a364-f70a1a4a62e8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:03.117565 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e4fa3-25f7-481f-8b87-7c47e70b1e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:03.118468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e4fa3-25f7-481f-8b87-7c47e70b1e8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:03.118753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e4fa3-25f7-481f-8b87-7c47e70b1e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:03.119031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e4fa3-25f7-481f-8b87-7c47e70b1e8e admin admin] PATH_INFO: `/v3/projects/9708db9615c94fcbb9a76ac33e240628/users/f2fc7b5c60dd41118d336d26eb32a276/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:03.121006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e4fa3-25f7-481f-8b87-7c47e70b1e8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9708db9615c94fcbb9a76ac33e240628, user_id=f2fc7b5c60dd41118d336d26eb32a276, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:03.123666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e4fa3-25f7-481f-8b87-7c47e70b1e8e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:03.130610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:03.147618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1033/2092] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:03 2022] PUT /identity/v3/projects/9708db9615c94fcbb9a76ac33e240628/users/f2fc7b5c60dd41118d336d26eb32a276/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:03.151794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c162a8-b956-4e3e-8a67-38f43bb40cb8 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:03.152087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c162a8-b956-4e3e-8a67-38f43bb40cb8 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:03.152365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c162a8-b956-4e3e-8a67-38f43bb40cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:03.179039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-62c162a8-b956-4e3e-8a67-38f43bb40cb8 None None] MFA Rules not processed for user `f2fc7b5c60dd41118d336d26eb32a276`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:03.180264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-62c162a8-b956-4e3e-8a67-38f43bb40cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:03.224928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:03.248887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1060/2093] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:03.260389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e4fa3-25f7-481f-8b87-7c47e70b1e8e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:03.262215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fb4e4fa3-25f7-481f-8b87-7c47e70b1e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:03.319446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e4fa3-25f7-481f-8b87-7c47e70b1e8e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:03.325332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d4c23-e4fd-4699-a834-01f0d0f152e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:03.325975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d4c23-e4fd-4699-a834-01f0d0f152e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:03.326284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d4c23-e4fd-4699-a834-01f0d0f152e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:03.326557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03d4c23-e4fd-4699-a834-01f0d0f152e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:03.382458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d4c23-e4fd-4699-a834-01f0d0f152e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:03.383879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03d4c23-e4fd-4699-a834-01f0d0f152e5 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:03.392495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1034/2094] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:13.245535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=8 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:13.388481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=49 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:14.919884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c162a8-b956-4e3e-8a67-38f43bb40cb8 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:14.929816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c162a8-b956-4e3e-8a67-38f43bb40cb8 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:14.934254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8342949c-1cf7-422f-9782-838337bcf3a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:14.934997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342949c-1cf7-422f-9782-838337bcf3a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:14.935281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342949c-1cf7-422f-9782-838337bcf3a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:14.935558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8342949c-1cf7-422f-9782-838337bcf3a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:14.945139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342949c-1cf7-422f-9782-838337bcf3a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:14.946593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8342949c-1cf7-422f-9782-838337bcf3a3 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:14.956240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1061/2095] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:24.951738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:25.044981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d4c23-e4fd-4699-a834-01f0d0f152e5 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:25.115828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03d4c23-e4fd-4699-a834-01f0d0f152e5 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:25.120436 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:25.121309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:25.122405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:25.124074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] PATH_INFO: `/v3/users/e8ffe1dee7e34bb7bcdfe01723f8267c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:25.126296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8ffe1dee7e34bb7bcdfe01723f8267c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:25.127820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:25.195096 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8ffe1dee7e34bb7bcdfe01723f8267c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.195894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8ffe1dee7e34bb7bcdfe01723f8267c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.196500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8ffe1dee7e34bb7bcdfe01723f8267c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.211218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8ffe1dee7e34bb7bcdfe01723f8267c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.221414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8ffe1dee7e34bb7bcdfe01723f8267c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.236467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1035/2096] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:25 2022] DELETE /identity/v3/users/e8ffe1dee7e34bb7bcdfe01723f8267c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:25.240890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5354-35fc-44ad-957c-d5132ad6044b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:25.241248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5354-35fc-44ad-957c-d5132ad6044b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:25.241688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1a5354-35fc-44ad-957c-d5132ad6044b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:25.266318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-da1a5354-35fc-44ad-957c-d5132ad6044b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:25.267834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-da1a5354-35fc-44ad-957c-d5132ad6044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:25.343431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1062/2097] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:25.353728 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:25.355404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:25.418809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a38750c-9350-4502-9c11-8ff537b68fe0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:25.423214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d07a6-a918-47c1-a205-6c330163c393 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:25.423865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d07a6-a918-47c1-a205-6c330163c393 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:25.424153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d07a6-a918-47c1-a205-6c330163c393 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:25.424432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424d07a6-a918-47c1-a205-6c330163c393 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:25.433342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d07a6-a918-47c1-a205-6c330163c393 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:25.434796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424d07a6-a918-47c1-a205-6c330163c393 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:25.444131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1036/2098] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:25.708694 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a5354-35fc-44ad-957c-d5132ad6044b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:25.710301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-da1a5354-35fc-44ad-957c-d5132ad6044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:25.724992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1a5354-35fc-44ad-957c-d5132ad6044b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:25.729374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:25.730420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:25.730732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:25.731515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] PATH_INFO: `/v3/projects/2a084cbaeffb4e44ba085664cc287943` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:25.732920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a084cbaeffb4e44ba085664cc287943)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:25.734748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:25.785451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a084cbaeffb4e44ba085664cc287943', 'request_id': 'req-abb8d91d-b500-438c-a813-19502b6fc73d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.790926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a084cbaeffb4e44ba085664cc287943', 'request_id': 'req-abb8d91d-b500-438c-a813-19502b6fc73d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.798175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a084cbaeffb4e44ba085664cc287943', 'request_id': 'req-abb8d91d-b500-438c-a813-19502b6fc73d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.809040 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] The token cache is being invalidate because project 2a084cbaeffb4e44ba085664cc287943 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:25.809355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.809977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.811531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1063/2099] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:25 2022] DELETE /identity/v3/projects/2a084cbaeffb4e44ba085664cc287943 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:25.815383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d07a6-a918-47c1-a205-6c330163c393 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:25.825093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424d07a6-a918-47c1-a205-6c330163c393 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:25.843675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:25.890673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:25.891590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:25.891834 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:25.892111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] PATH_INFO: `/v3/users/f2fc7b5c60dd41118d336d26eb32a276` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:25.894235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2fc7b5c60dd41118d336d26eb32a276)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:25.896151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:25.947740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2fc7b5c60dd41118d336d26eb32a276'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.948340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2fc7b5c60dd41118d336d26eb32a276'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.952454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2fc7b5c60dd41118d336d26eb32a276'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.967171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2fc7b5c60dd41118d336d26eb32a276'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.976831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-255a32db-8cb3-4b99-b461-032356b92b50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2fc7b5c60dd41118d336d26eb32a276'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:25.998005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1037/2100] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:25 2022] DELETE /identity/v3/users/f2fc7b5c60dd41118d336d26eb32a276 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:26.332028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:26.334025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:26.346914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb8d91d-b500-438c-a813-19502b6fc73d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:26.408202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:26.409076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:26.409358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:26.409635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] PATH_INFO: `/v3/projects/9708db9615c94fcbb9a76ac33e240628` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:26.410872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9708db9615c94fcbb9a76ac33e240628)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:26.413275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:26.445197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:26.464686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9708db9615c94fcbb9a76ac33e240628', 'request_id': 'req-0eb32716-e056-42f7-b0d8-c184aa72b450'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:26.470840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9708db9615c94fcbb9a76ac33e240628', 'request_id': 'req-0eb32716-e056-42f7-b0d8-c184aa72b450'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:26.478552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9708db9615c94fcbb9a76ac33e240628', 'request_id': 'req-0eb32716-e056-42f7-b0d8-c184aa72b450'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:26.491370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] The token cache is being invalidate because project 9708db9615c94fcbb9a76ac33e240628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:26.491663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:26.492319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0eb32716-e056-42f7-b0d8-c184aa72b450 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:26.493878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1064/2101] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:26 2022] DELETE /identity/v3/projects/9708db9615c94fcbb9a76ac33e240628 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:26.500285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9e9b2-30ed-4793-833e-2c9e46fc7771 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:26.500753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9e9b2-30ed-4793-833e-2c9e46fc7771 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:26.500888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded9e9b2-30ed-4793-833e-2c9e46fc7771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:26.518367 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ded9e9b2-30ed-4793-833e-2c9e46fc7771 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:26.519558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ded9e9b2-30ed-4793-833e-2c9e46fc7771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:26.594057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1038/2102] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:26.597308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c5fa8-b7af-40f5-9219-958bc7cdc60e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:26.597537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c5fa8-b7af-40f5-9219-958bc7cdc60e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:26.597815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c5fa8-b7af-40f5-9219-958bc7cdc60e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:26.616359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-624c5fa8-b7af-40f5-9219-958bc7cdc60e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:26.617677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-624c5fa8-b7af-40f5-9219-958bc7cdc60e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:26.628454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1065/2103] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:26.632735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9e9b2-30ed-4793-833e-2c9e46fc7771 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:26.642407 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded9e9b2-30ed-4793-833e-2c9e46fc7771 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:26.647057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc8dcf-ad05-4dba-8eff-e966efcae4eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:26.647704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc8dcf-ad05-4dba-8eff-e966efcae4eb admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:26.647993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc8dcf-ad05-4dba-8eff-e966efcae4eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:26.648272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc8dcf-ad05-4dba-8eff-e966efcae4eb admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:26.649010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc8dcf-ad05-4dba-8eff-e966efcae4eb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:26.649321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc8dcf-ad05-4dba-8eff-e966efcae4eb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:26.650959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc8dcf-ad05-4dba-8eff-e966efcae4eb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:26.662995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1039/2104] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:06:26.667433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c5fa8-b7af-40f5-9219-958bc7cdc60e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:26.679056 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c5fa8-b7af-40f5-9219-958bc7cdc60e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:26.684017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870ef71-ae8b-4784-b00e-8dda00791ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:26.684667 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870ef71-ae8b-4784-b00e-8dda00791ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:26.684957 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870ef71-ae8b-4784-b00e-8dda00791ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:26.685243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870ef71-ae8b-4784-b00e-8dda00791ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:26.685926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870ef71-ae8b-4784-b00e-8dda00791ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:26.687889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9870ef71-ae8b-4784-b00e-8dda00791ceb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:26.713266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1066/2105] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:26 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:26.717922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc8dcf-ad05-4dba-8eff-e966efcae4eb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:26.728903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc8dcf-ad05-4dba-8eff-e966efcae4eb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:26.757012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:26.801094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88656024-acf5-4fb6-ba56-2adafddc50ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:26.801912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88656024-acf5-4fb6-ba56-2adafddc50ef admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:26.802296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88656024-acf5-4fb6-ba56-2adafddc50ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:26.802610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88656024-acf5-4fb6-ba56-2adafddc50ef admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:26.803503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88656024-acf5-4fb6-ba56-2adafddc50ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:26.805162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88656024-acf5-4fb6-ba56-2adafddc50ef admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:26.830567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1040/2106] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:26 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:26.834640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870ef71-ae8b-4784-b00e-8dda00791ceb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:26.847448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870ef71-ae8b-4784-b00e-8dda00791ceb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:26.857481 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83734816-0147-4a59-b003-4b8e6f4eefb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:26.858931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83734816-0147-4a59-b003-4b8e6f4eefb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:26.859522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83734816-0147-4a59-b003-4b8e6f4eefb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:26.859640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83734816-0147-4a59-b003-4b8e6f4eefb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:26.860811 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83734816-0147-4a59-b003-4b8e6f4eefb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:26.863535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83734816-0147-4a59-b003-4b8e6f4eefb5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:26.881573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1067/2107] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:26.886101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88656024-acf5-4fb6-ba56-2adafddc50ef admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:26.896077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88656024-acf5-4fb6-ba56-2adafddc50ef admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:26.900590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b758de0e-7325-4551-9cd9-71bac9cb8770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:26.901236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758de0e-7325-4551-9cd9-71bac9cb8770 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:26.901519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758de0e-7325-4551-9cd9-71bac9cb8770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:26.901795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b758de0e-7325-4551-9cd9-71bac9cb8770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:26.902548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758de0e-7325-4551-9cd9-71bac9cb8770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:26.904131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b758de0e-7325-4551-9cd9-71bac9cb8770 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:26.914734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1041/2108] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:26.918922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83734816-0147-4a59-b003-4b8e6f4eefb5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:26.929602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83734816-0147-4a59-b003-4b8e6f4eefb5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:26.934168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9f15-cebb-40b3-a79c-23863b2d78a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:26.935072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9f15-cebb-40b3-a79c-23863b2d78a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:26.935357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9f15-cebb-40b3-a79c-23863b2d78a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:26.935635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adf9f15-cebb-40b3-a79c-23863b2d78a6 admin admin] PATH_INFO: `/v3/projects/1f1dc43e580e4707afcf63a0f8e719c7/users/279b28abb83747f8a25f7f02f3aa8d21/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:26.949761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf9f15-cebb-40b3-a79c-23863b2d78a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f1dc43e580e4707afcf63a0f8e719c7, user_id=279b28abb83747f8a25f7f02f3aa8d21, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:26.952478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adf9f15-cebb-40b3-a79c-23863b2d78a6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:26.965532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1068/2109] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:26 2022] PUT /identity/v3/projects/1f1dc43e580e4707afcf63a0f8e719c7/users/279b28abb83747f8a25f7f02f3aa8d21/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:26.978516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820f876-f218-4be5-a86d-b14a393cb5bd None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:26.978809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820f876-f218-4be5-a86d-b14a393cb5bd None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:26.979089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820f876-f218-4be5-a86d-b14a393cb5bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:27.007830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c820f876-f218-4be5-a86d-b14a393cb5bd None None] MFA Rules not processed for user `279b28abb83747f8a25f7f02f3aa8d21`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:27.009026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c820f876-f218-4be5-a86d-b14a393cb5bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:27.086649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1042/2110] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:27.095611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9f15-cebb-40b3-a79c-23863b2d78a6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:27.097441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3adf9f15-cebb-40b3-a79c-23863b2d78a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:27.174147 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adf9f15-cebb-40b3-a79c-23863b2d78a6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:27.181915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bad46c-e7ad-4265-98e3-d6feae62617f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:27.181915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bad46c-e7ad-4265-98e3-d6feae62617f service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:27.181915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bad46c-e7ad-4265-98e3-d6feae62617f service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:27.181915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bad46c-e7ad-4265-98e3-d6feae62617f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:27.202718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bad46c-e7ad-4265-98e3-d6feae62617f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:27.204482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bad46c-e7ad-4265-98e3-d6feae62617f service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:27.217756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1069/2111] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:27.245878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820f876-f218-4be5-a86d-b14a393cb5bd None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:27.337995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bad46c-e7ad-4265-98e3-d6feae62617f service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:27.339604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f9bad46c-e7ad-4265-98e3-d6feae62617f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:27.362396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820f876-f218-4be5-a86d-b14a393cb5bd None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:27.367194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:27.368323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:27.368799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:27.369357 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] PATH_INFO: `/v3/users/279b28abb83747f8a25f7f02f3aa8d21` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:27.371278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=279b28abb83747f8a25f7f02f3aa8d21)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:27.373960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:27.439780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:27.457729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:27.469086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '279b28abb83747f8a25f7f02f3aa8d21'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:27.470351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '279b28abb83747f8a25f7f02f3aa8d21'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:27.471272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '279b28abb83747f8a25f7f02f3aa8d21'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:27.473614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bad46c-e7ad-4265-98e3-d6feae62617f service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:27.480904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:27.482216 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:27.482505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:27.483003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] PATH_INFO: `/v3/users/40e7ab96a0e34a1ab02e7243362ca215` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:27.484536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=40e7ab96a0e34a1ab02e7243362ca215)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:27.487642 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:27.491856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '279b28abb83747f8a25f7f02f3aa8d21'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:27.506392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4d49ee4d-5499-4d24-92c0-cfb286f9ca1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '279b28abb83747f8a25f7f02f3aa8d21'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:27.527288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1043/2112] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:27 2022] DELETE /identity/v3/users/279b28abb83747f8a25f7f02f3aa8d21 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:27.533059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b964176-3551-4319-8cd3-2a4c424294ee None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:27.533344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b964176-3551-4319-8cd3-2a4c424294ee None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:27.533755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b964176-3551-4319-8cd3-2a4c424294ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:27.551836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9b964176-3551-4319-8cd3-2a4c424294ee None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:27.554166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9b964176-3551-4319-8cd3-2a4c424294ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:27.559187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40e7ab96a0e34a1ab02e7243362ca215'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:27.560022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40e7ab96a0e34a1ab02e7243362ca215'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:27.568422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40e7ab96a0e34a1ab02e7243362ca215'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:27.581137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40e7ab96a0e34a1ab02e7243362ca215'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:27.582289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-1fc30034-b44c-427d-85af-167df15d2a7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40e7ab96a0e34a1ab02e7243362ca215'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:27.618119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1070/2113] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:27 2022] DELETE /identity/v3/users/40e7ab96a0e34a1ab02e7243362ca215 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:27.620337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409e78f-37f3-47df-a028-868f6656fbe0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:27.622978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409e78f-37f3-47df-a028-868f6656fbe0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:27.624336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0409e78f-37f3-47df-a028-868f6656fbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:27.658677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-0409e78f-37f3-47df-a028-868f6656fbe0 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:27.658677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0409e78f-37f3-47df-a028-868f6656fbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:27.662792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1044/2114] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:27.674904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b964176-3551-4319-8cd3-2a4c424294ee None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:27.677156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9b964176-3551-4319-8cd3-2a4c424294ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:27.728952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1071/2115] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:27.739296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409e78f-37f3-47df-a028-868f6656fbe0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:27.740793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0409e78f-37f3-47df-a028-868f6656fbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:27.750998 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b964176-3551-4319-8cd3-2a4c424294ee None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:27.804919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0409e78f-37f3-47df-a028-868f6656fbe0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:27.805184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa046dd-5729-45c0-9780-88c8d5f3c7cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:27.805562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa046dd-5729-45c0-9780-88c8d5f3c7cb service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:27.805846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa046dd-5729-45c0-9780-88c8d5f3c7cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:27.806153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa046dd-5729-45c0-9780-88c8d5f3c7cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:27.808688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eebc08-6433-4605-a06b-8bfa10c8e059 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:27.809330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eebc08-6433-4605-a06b-8bfa10c8e059 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:27.809612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eebc08-6433-4605-a06b-8bfa10c8e059 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:27.809888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eebc08-6433-4605-a06b-8bfa10c8e059 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:27.815697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa046dd-5729-45c0-9780-88c8d5f3c7cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:27.817114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa046dd-5729-45c0-9780-88c8d5f3c7cb service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:27.819587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eebc08-6433-4605-a06b-8bfa10c8e059 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:27.820998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eebc08-6433-4605-a06b-8bfa10c8e059 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:27.826520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1045/2116] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:27.830481 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1072/2117] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:06:28.185858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa046dd-5729-45c0-9780-88c8d5f3c7cb service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.188325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4aa046dd-5729-45c0-9780-88c8d5f3c7cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:28.201587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa046dd-5729-45c0-9780-88c8d5f3c7cb service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.206306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.207103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.207422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.207671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] PATH_INFO: `/v3/projects/1f1dc43e580e4707afcf63a0f8e719c7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.211362 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f1dc43e580e4707afcf63a0f8e719c7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.218109 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.230369 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:28.230511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eebc08-6433-4605-a06b-8bfa10c8e059 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.232139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b3eebc08-6433-4605-a06b-8bfa10c8e059 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:28.246965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eebc08-6433-4605-a06b-8bfa10c8e059 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.252932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.254179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.254593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.254721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] PATH_INFO: `/v3/projects/3851eeeb702a4986bd91da32153fbb36` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.255891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3851eeeb702a4986bd91da32153fbb36)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.257794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.270355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:28.277921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f1dc43e580e4707afcf63a0f8e719c7', 'request_id': 'req-9fe324ad-e888-4482-b058-746dfe96a7eb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:28.286321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f1dc43e580e4707afcf63a0f8e719c7', 'request_id': 'req-9fe324ad-e888-4482-b058-746dfe96a7eb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:28.292975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f1dc43e580e4707afcf63a0f8e719c7', 'request_id': 'req-9fe324ad-e888-4482-b058-746dfe96a7eb'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:28.305055 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] The token cache is being invalidate because project 1f1dc43e580e4707afcf63a0f8e719c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:28.305417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:28.306134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9fe324ad-e888-4482-b058-746dfe96a7eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:28.307757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1046/2118] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:28 2022] DELETE /identity/v3/projects/1f1dc43e580e4707afcf63a0f8e719c7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:28.312040 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3851eeeb702a4986bd91da32153fbb36', 'request_id': 'req-a06cf84f-f098-4a91-a2d1-967a449d196f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:28.312656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3c7f6-1a42-4f33-af7f-fa7ed9676f43 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.312957 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3c7f6-1a42-4f33-af7f-fa7ed9676f43 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.313242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3c7f6-1a42-4f33-af7f-fa7ed9676f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.318824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3851eeeb702a4986bd91da32153fbb36', 'request_id': 'req-a06cf84f-f098-4a91-a2d1-967a449d196f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:28.328652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3851eeeb702a4986bd91da32153fbb36', 'request_id': 'req-a06cf84f-f098-4a91-a2d1-967a449d196f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:28.332447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-35a3c7f6-1a42-4f33-af7f-fa7ed9676f43 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:28.333674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-35a3c7f6-1a42-4f33-af7f-fa7ed9676f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:28.341903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] The token cache is being invalidate because project 3851eeeb702a4986bd91da32153fbb36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:28.342944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:28.343682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a06cf84f-f098-4a91-a2d1-967a449d196f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:28.346280 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1073/2119] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:28 2022] DELETE /identity/v3/projects/3851eeeb702a4986bd91da32153fbb36 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:28.351175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ea194-6e67-473a-9abb-b9a1c854110f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.351471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ea194-6e67-473a-9abb-b9a1c854110f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.351757 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9ea194-6e67-473a-9abb-b9a1c854110f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.373822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1a9ea194-6e67-473a-9abb-b9a1c854110f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:28.375082 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1a9ea194-6e67-473a-9abb-b9a1c854110f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:28.402249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1047/2120] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:28.411389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e53c18-fdc0-496a-8469-0a923b3f5c44 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.411836 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e53c18-fdc0-496a-8469-0a923b3f5c44 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.412584 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e53c18-fdc0-496a-8469-0a923b3f5c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.430901 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d1e53c18-fdc0-496a-8469-0a923b3f5c44 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:28.432235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d1e53c18-fdc0-496a-8469-0a923b3f5c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:28.446970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1048/2121] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:28.447156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1074/2122] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:28.448138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e53c18-fdc0-496a-8469-0a923b3f5c44 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.449034 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc394505-5c8c-4463-ac71-2f425e8f1c53 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.449393 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc394505-5c8c-4463-ac71-2f425e8f1c53 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.449742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc394505-5c8c-4463-ac71-2f425e8f1c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.457722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e53c18-fdc0-496a-8469-0a923b3f5c44 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.463705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682690e-a939-4d4e-996f-f8fa0f6e66df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.464631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682690e-a939-4d4e-996f-f8fa0f6e66df admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.464988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682690e-a939-4d4e-996f-f8fa0f6e66df admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.465370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2682690e-a939-4d4e-996f-f8fa0f6e66df admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.466189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682690e-a939-4d4e-996f-f8fa0f6e66df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:28.466571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682690e-a939-4d4e-996f-f8fa0f6e66df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.469277 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2682690e-a939-4d4e-996f-f8fa0f6e66df admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.469853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dc394505-5c8c-4463-ac71-2f425e8f1c53 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:28.471251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dc394505-5c8c-4463-ac71-2f425e8f1c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:28.479427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1075/2123] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:28.483833 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc394505-5c8c-4463-ac71-2f425e8f1c53 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.487437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1049/2124] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:06:28.488404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682690e-a939-4d4e-996f-f8fa0f6e66df admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.493613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc394505-5c8c-4463-ac71-2f425e8f1c53 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.498833 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f73224-edd0-439a-a1d1-db1cb8b2f492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.499496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f73224-edd0-439a-a1d1-db1cb8b2f492 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.499781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f73224-edd0-439a-a1d1-db1cb8b2f492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.500058 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f73224-edd0-439a-a1d1-db1cb8b2f492 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.501204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2682690e-a939-4d4e-996f-f8fa0f6e66df admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.501308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f73224-edd0-439a-a1d1-db1cb8b2f492 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:28.501396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f73224-edd0-439a-a1d1-db1cb8b2f492 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.503013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f73224-edd0-439a-a1d1-db1cb8b2f492 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.506074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc264a89-6687-46e6-b668-5c1ae00a53e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.506273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc264a89-6687-46e6-b668-5c1ae00a53e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.506558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc264a89-6687-46e6-b668-5c1ae00a53e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.507096 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc264a89-6687-46e6-b668-5c1ae00a53e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.507508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc264a89-6687-46e6-b668-5c1ae00a53e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.509439 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc264a89-6687-46e6-b668-5c1ae00a53e2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.515740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1076/2125] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:06:28.519728 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f73224-edd0-439a-a1d1-db1cb8b2f492 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.530407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f73224-edd0-439a-a1d1-db1cb8b2f492 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.535100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c134a2-3dc1-478c-b65d-72b8e5d731ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.535765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c134a2-3dc1-478c-b65d-72b8e5d731ff admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.536050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c134a2-3dc1-478c-b65d-72b8e5d731ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.536328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c134a2-3dc1-478c-b65d-72b8e5d731ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.537015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c134a2-3dc1-478c-b65d-72b8e5d731ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.537494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1050/2126] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:28.539132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c134a2-3dc1-478c-b65d-72b8e5d731ff admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.540722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc264a89-6687-46e6-b668-5c1ae00a53e2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.568180 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1077/2127] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:28.569253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c134a2-3dc1-478c-b65d-72b8e5d731ff admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.600112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:28.604247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc264a89-6687-46e6-b668-5c1ae00a53e2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.627524 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:28.631597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c134a2-3dc1-478c-b65d-72b8e5d731ff admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.637289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29206c-a0a8-4a44-9d5e-f9db944289b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.637981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29206c-a0a8-4a44-9d5e-f9db944289b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.638303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29206c-a0a8-4a44-9d5e-f9db944289b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.638758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29206c-a0a8-4a44-9d5e-f9db944289b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.640088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29206c-a0a8-4a44-9d5e-f9db944289b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.642161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29206c-a0a8-4a44-9d5e-f9db944289b3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.662463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1078/2128] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:28.665014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdace36f-385e-41cf-b628-7d74cbba900d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.665698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdace36f-385e-41cf-b628-7d74cbba900d admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.666027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdace36f-385e-41cf-b628-7d74cbba900d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.666314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29206c-a0a8-4a44-9d5e-f9db944289b3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.666426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdace36f-385e-41cf-b628-7d74cbba900d admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.667022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdace36f-385e-41cf-b628-7d74cbba900d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.669452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdace36f-385e-41cf-b628-7d74cbba900d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.678618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29206c-a0a8-4a44-9d5e-f9db944289b3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.685383 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa68b7-0a22-47ce-89e2-832c6a48f482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.686120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa68b7-0a22-47ce-89e2-832c6a48f482 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.686408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa68b7-0a22-47ce-89e2-832c6a48f482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.686685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa68b7-0a22-47ce-89e2-832c6a48f482 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.687413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa68b7-0a22-47ce-89e2-832c6a48f482 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.689005 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa68b7-0a22-47ce-89e2-832c6a48f482 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.694924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1051/2129] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:28.696210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdace36f-385e-41cf-b628-7d74cbba900d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.704707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1079/2130] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:28.705623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa68b7-0a22-47ce-89e2-832c6a48f482 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.707706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdace36f-385e-41cf-b628-7d74cbba900d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.712341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dce3ea-ce60-4864-beda-8a937307c5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.713001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce3ea-ce60-4864-beda-8a937307c5c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.713281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce3ea-ce60-4864-beda-8a937307c5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.713552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dce3ea-ce60-4864-beda-8a937307c5c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.714332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dce3ea-ce60-4864-beda-8a937307c5c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.715928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dce3ea-ce60-4864-beda-8a937307c5c8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.716033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa68b7-0a22-47ce-89e2-832c6a48f482 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.722173 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df8447-3418-4899-85a3-5ad1601ac38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.723073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df8447-3418-4899-85a3-5ad1601ac38f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.723359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df8447-3418-4899-85a3-5ad1601ac38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.723631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df8447-3418-4899-85a3-5ad1601ac38f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.724360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df8447-3418-4899-85a3-5ad1601ac38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.725942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df8447-3418-4899-85a3-5ad1601ac38f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.732851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1052/2131] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:28.735016 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dce3ea-ce60-4864-beda-8a937307c5c8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.742518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1080/2132] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:28.744022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df8447-3418-4899-85a3-5ad1601ac38f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.745825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dce3ea-ce60-4864-beda-8a937307c5c8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.751778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4a1a8-1354-44e2-81bb-48947bdb3b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.752456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4a1a8-1354-44e2-81bb-48947bdb3b86 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.752744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4a1a8-1354-44e2-81bb-48947bdb3b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.753045 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f4a1a8-1354-44e2-81bb-48947bdb3b86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.754061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4a1a8-1354-44e2-81bb-48947bdb3b86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.754423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df8447-3418-4899-85a3-5ad1601ac38f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.755681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f4a1a8-1354-44e2-81bb-48947bdb3b86 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.759844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655d8b7-255e-4b56-9853-1c8a9f7c94c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.760892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655d8b7-255e-4b56-9853-1c8a9f7c94c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.761183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655d8b7-255e-4b56-9853-1c8a9f7c94c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.761449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655d8b7-255e-4b56-9853-1c8a9f7c94c4 admin admin] PATH_INFO: `/v3/projects/1e93a6fa3aa94857bde480c34b0051aa/users/a4603fc7a991497e8757dca042e8df23/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.772114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1053/2133] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:28.776436 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4a1a8-1354-44e2-81bb-48947bdb3b86 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.785767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655d8b7-255e-4b56-9853-1c8a9f7c94c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e93a6fa3aa94857bde480c34b0051aa, user_id=a4603fc7a991497e8757dca042e8df23, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.788615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655d8b7-255e-4b56-9853-1c8a9f7c94c4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.790960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f4a1a8-1354-44e2-81bb-48947bdb3b86 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:28.801336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39c69b6-d700-4133-b983-aac0be08aec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:28.802430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c69b6-d700-4133-b983-aac0be08aec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.802724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c69b6-d700-4133-b983-aac0be08aec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.803031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c69b6-d700-4133-b983-aac0be08aec7 admin admin] PATH_INFO: `/v3/projects/841135cad3e24ddd92df811779f6595a/users/84b99364e5fb439dbfbce21eb9bac786/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.815001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1081/2134] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:28 2022] PUT /identity/v3/projects/1e93a6fa3aa94857bde480c34b0051aa/users/a4603fc7a991497e8757dca042e8df23/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:28.818171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bff1a1-300f-4d84-9d27-1e5acb8e5bde None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.818464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bff1a1-300f-4d84-9d27-1e5acb8e5bde None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.818738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bff1a1-300f-4d84-9d27-1e5acb8e5bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.819283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c69b6-d700-4133-b983-aac0be08aec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=841135cad3e24ddd92df811779f6595a, user_id=84b99364e5fb439dbfbce21eb9bac786, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:28.821966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39c69b6-d700-4133-b983-aac0be08aec7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:28.858450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1054/2135] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:28 2022] PUT /identity/v3/projects/841135cad3e24ddd92df811779f6595a/users/84b99364e5fb439dbfbce21eb9bac786/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:28.860208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84549782-321a-4ae6-9feb-37607457df45 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:28.860521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84549782-321a-4ae6-9feb-37607457df45 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:28.860800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84549782-321a-4ae6-9feb-37607457df45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:28.871596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b4bff1a1-300f-4d84-9d27-1e5acb8e5bde None None] MFA Rules not processed for user `a4603fc7a991497e8757dca042e8df23`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:28.872932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b4bff1a1-300f-4d84-9d27-1e5acb8e5bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:28.900189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-84549782-321a-4ae6-9feb-37607457df45 None None] MFA Rules not processed for user `84b99364e5fb439dbfbce21eb9bac786`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:28.901606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-84549782-321a-4ae6-9feb-37607457df45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:28.948997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1082/2136] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:28.954979 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bff1a1-300f-4d84-9d27-1e5acb8e5bde None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.982380 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1055/2137] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:28.994406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84549782-321a-4ae6-9feb-37607457df45 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:28.996064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-84549782-321a-4ae6-9feb-37607457df45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:29.037150 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bff1a1-300f-4d84-9d27-1e5acb8e5bde None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.042523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:29.043585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588973e1-982c-4bd5-a1ae-ebe6403272f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.044558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588973e1-982c-4bd5-a1ae-ebe6403272f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.044885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588973e1-982c-4bd5-a1ae-ebe6403272f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.045780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588973e1-982c-4bd5-a1ae-ebe6403272f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.046164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588973e1-982c-4bd5-a1ae-ebe6403272f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.048565 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588973e1-982c-4bd5-a1ae-ebe6403272f2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.061529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84549782-321a-4ae6-9feb-37607457df45 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.065817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14678e-aa39-4b2d-84f9-7dd02e5058b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.066777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14678e-aa39-4b2d-84f9-7dd02e5058b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.067171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14678e-aa39-4b2d-84f9-7dd02e5058b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.067573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f14678e-aa39-4b2d-84f9-7dd02e5058b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.077953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14678e-aa39-4b2d-84f9-7dd02e5058b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.081505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1083/2138] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:28 2022] POST /identity/v3/projects => generated 383 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:29.081638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f14678e-aa39-4b2d-84f9-7dd02e5058b1 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.082816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588973e1-982c-4bd5-a1ae-ebe6403272f2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:29.089517 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1056/2139] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:28 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:06:29.105975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:29.158364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588973e1-982c-4bd5-a1ae-ebe6403272f2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.163066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50339c-6040-4c82-a5a8-bebd1a458e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.163757 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50339c-6040-4c82-a5a8-bebd1a458e29 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.164027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50339c-6040-4c82-a5a8-bebd1a458e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.164343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac50339c-6040-4c82-a5a8-bebd1a458e29 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.164992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50339c-6040-4c82-a5a8-bebd1a458e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.170484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac50339c-6040-4c82-a5a8-bebd1a458e29 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.195537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1084/2140] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:29 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:29.199445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14678e-aa39-4b2d-84f9-7dd02e5058b1 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:29.215455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f14678e-aa39-4b2d-84f9-7dd02e5058b1 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.221476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa75b5e-c793-4ea2-a3c8-1bcc10ce231c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.222252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa75b5e-c793-4ea2-a3c8-1bcc10ce231c admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.222604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa75b5e-c793-4ea2-a3c8-1bcc10ce231c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.222888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa75b5e-c793-4ea2-a3c8-1bcc10ce231c admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.223657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa75b5e-c793-4ea2-a3c8-1bcc10ce231c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.225265 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa75b5e-c793-4ea2-a3c8-1bcc10ce231c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.236681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1057/2141] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:29.240475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50339c-6040-4c82-a5a8-bebd1a458e29 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:29.252931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac50339c-6040-4c82-a5a8-bebd1a458e29 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.257904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbffe9d-ff99-4f31-aa42-7d3b005e218a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.258611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbffe9d-ff99-4f31-aa42-7d3b005e218a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.258898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbffe9d-ff99-4f31-aa42-7d3b005e218a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.259178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbffe9d-ff99-4f31-aa42-7d3b005e218a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.259905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbffe9d-ff99-4f31-aa42-7d3b005e218a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.261502 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbffe9d-ff99-4f31-aa42-7d3b005e218a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.273995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1085/2142] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:29.278129 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa75b5e-c793-4ea2-a3c8-1bcc10ce231c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:29.291175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa75b5e-c793-4ea2-a3c8-1bcc10ce231c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.298190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4de70-2e08-4c01-aaab-a7be8b4a457a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.299579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4de70-2e08-4c01-aaab-a7be8b4a457a admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.300046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4de70-2e08-4c01-aaab-a7be8b4a457a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.300493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d4de70-2e08-4c01-aaab-a7be8b4a457a admin admin] PATH_INFO: `/v3/projects/243bb5851f5849fdab6410f4c114f4e2/users/1b8a8dce306b49278589b8fef1973650/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.320613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d4de70-2e08-4c01-aaab-a7be8b4a457a admin admin] RBAC: Authorizing `identity:create_grant(project_id=243bb5851f5849fdab6410f4c114f4e2, user_id=1b8a8dce306b49278589b8fef1973650, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.324624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d4de70-2e08-4c01-aaab-a7be8b4a457a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.341912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1058/2143] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:29 2022] PUT /identity/v3/projects/243bb5851f5849fdab6410f4c114f4e2/users/1b8a8dce306b49278589b8fef1973650/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:29.356689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053db18-fe79-4e19-9cbb-2b6a1230eaaa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.356689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053db18-fe79-4e19-9cbb-2b6a1230eaaa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.356689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c053db18-fe79-4e19-9cbb-2b6a1230eaaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.385755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c053db18-fe79-4e19-9cbb-2b6a1230eaaa None None] MFA Rules not processed for user `1b8a8dce306b49278589b8fef1973650`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:29.387204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c053db18-fe79-4e19-9cbb-2b6a1230eaaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:29.474402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1086/2144] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:29.480190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4de70-2e08-4c01-aaab-a7be8b4a457a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:29.599471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d4de70-2e08-4c01-aaab-a7be8b4a457a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.603832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da95f79-f616-46df-8339-379841f5e592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.604738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da95f79-f616-46df-8339-379841f5e592 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.605086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da95f79-f616-46df-8339-379841f5e592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.605971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da95f79-f616-46df-8339-379841f5e592 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.606216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da95f79-f616-46df-8339-379841f5e592 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.609184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da95f79-f616-46df-8339-379841f5e592 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.640662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1059/2145] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:29 2022] POST /identity/v3/projects => generated 383 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:29.642074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053db18-fe79-4e19-9cbb-2b6a1230eaaa None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:29.652902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c053db18-fe79-4e19-9cbb-2b6a1230eaaa None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.714239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370381ef-836b-4f20-8d2e-cc3b296421f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.714970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370381ef-836b-4f20-8d2e-cc3b296421f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.715262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370381ef-836b-4f20-8d2e-cc3b296421f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.715538 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370381ef-836b-4f20-8d2e-cc3b296421f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.716227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370381ef-836b-4f20-8d2e-cc3b296421f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.718742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370381ef-836b-4f20-8d2e-cc3b296421f5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.741683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1087/2146] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:29 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:29.745713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da95f79-f616-46df-8339-379841f5e592 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:29.757996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da95f79-f616-46df-8339-379841f5e592 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.763432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91291295-2600-4a78-ac2a-7f215f22a01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.764155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91291295-2600-4a78-ac2a-7f215f22a01f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.764476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91291295-2600-4a78-ac2a-7f215f22a01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.764759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91291295-2600-4a78-ac2a-7f215f22a01f admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.765506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91291295-2600-4a78-ac2a-7f215f22a01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.767176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91291295-2600-4a78-ac2a-7f215f22a01f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.779111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1060/2147] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:29.784244 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370381ef-836b-4f20-8d2e-cc3b296421f5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:29.796068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370381ef-836b-4f20-8d2e-cc3b296421f5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.800905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb432560-22fd-42b4-8735-546562a256e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.801821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb432560-22fd-42b4-8735-546562a256e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.802142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb432560-22fd-42b4-8735-546562a256e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.802429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb432560-22fd-42b4-8735-546562a256e6 admin admin] PATH_INFO: `/v3/projects/3b510776db304275838a44767e366ead/users/770f4f71bbff4f7fa1209a3a4f7e2836/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.815028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:29.819724 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb432560-22fd-42b4-8735-546562a256e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b510776db304275838a44767e366ead, user_id=770f4f71bbff4f7fa1209a3a4f7e2836, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.823601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb432560-22fd-42b4-8735-546562a256e6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.838866 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1088/2148] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:29 2022] PUT /identity/v3/projects/3b510776db304275838a44767e366ead/users/770f4f71bbff4f7fa1209a3a4f7e2836/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:29.844217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91291295-2600-4a78-ac2a-7f215f22a01f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:29.863980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91291295-2600-4a78-ac2a-7f215f22a01f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.927064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae24f23-6713-466f-bfa1-344b6aaed9f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.927738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae24f23-6713-466f-bfa1-344b6aaed9f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.928028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae24f23-6713-466f-bfa1-344b6aaed9f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.928312 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae24f23-6713-466f-bfa1-344b6aaed9f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.929047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae24f23-6713-466f-bfa1-344b6aaed9f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.930685 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae24f23-6713-466f-bfa1-344b6aaed9f0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.941026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1061/2149] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:29.945148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb432560-22fd-42b4-8735-546562a256e6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:29.955593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb432560-22fd-42b4-8735-546562a256e6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:29.960478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f17359-d30a-466b-a050-59487cca83cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:29.961385 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f17359-d30a-466b-a050-59487cca83cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.961803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f17359-d30a-466b-a050-59487cca83cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.962217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f17359-d30a-466b-a050-59487cca83cd admin admin] PATH_INFO: `/v3/projects/3b510776db304275838a44767e366ead/users/770f4f71bbff4f7fa1209a3a4f7e2836/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:29.964312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f17359-d30a-466b-a050-59487cca83cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b510776db304275838a44767e366ead, user_id=770f4f71bbff4f7fa1209a3a4f7e2836, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:29.967580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f17359-d30a-466b-a050-59487cca83cd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:29.980207 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1089/2150] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:29 2022] PUT /identity/v3/projects/3b510776db304275838a44767e366ead/users/770f4f71bbff4f7fa1209a3a4f7e2836/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:29.991894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1c084-7113-4118-9264-82e1dff583c2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:29.992112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1c084-7113-4118-9264-82e1dff583c2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:29.992396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca1c084-7113-4118-9264-82e1dff583c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:30.025098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cca1c084-7113-4118-9264-82e1dff583c2 None None] MFA Rules not processed for user `770f4f71bbff4f7fa1209a3a4f7e2836`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:30.026596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cca1c084-7113-4118-9264-82e1dff583c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:30.031420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:30.103392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1062/2151] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:30.117971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f17359-d30a-466b-a050-59487cca83cd admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:30.119856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d2f17359-d30a-466b-a050-59487cca83cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:30.182998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f17359-d30a-466b-a050-59487cca83cd admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:30.186715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7412641-c527-4ce7-9bb0-e63c6cdd55e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:30.187387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7412641-c527-4ce7-9bb0-e63c6cdd55e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:30.187669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7412641-c527-4ce7-9bb0-e63c6cdd55e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:30.188292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7412641-c527-4ce7-9bb0-e63c6cdd55e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:30.247486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7412641-c527-4ce7-9bb0-e63c6cdd55e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:30.248864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7412641-c527-4ce7-9bb0-e63c6cdd55e8 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:30.258428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1090/2152] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:31.353171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c148-fd7a-44b2-bb38-93ab258b009a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:31.353700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c148-fd7a-44b2-bb38-93ab258b009a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:31.353781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a9c148-fd7a-44b2-bb38-93ab258b009a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:31.372559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-69a9c148-fd7a-44b2-bb38-93ab258b009a None None] MFA Rules not processed for user `c31dbb3dcd9a411ea125573c8189e00c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:31.374338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-69a9c148-fd7a-44b2-bb38-93ab258b009a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:31.433400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1063/2153] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:31.785647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7412641-c527-4ce7-9bb0-e63c6cdd55e8 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:31.794117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7412641-c527-4ce7-9bb0-e63c6cdd55e8 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:31.797761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f151c-2b85-4569-a538-0909a785b735 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:31.798515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f151c-2b85-4569-a538-0909a785b735 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:31.798864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f151c-2b85-4569-a538-0909a785b735 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:31.799199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f151c-2b85-4569-a538-0909a785b735 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:31.863206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f151c-2b85-4569-a538-0909a785b735 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:31.864758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0f151c-2b85-4569-a538-0909a785b735 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:31.881629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1091/2154] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:32.499453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9c148-fd7a-44b2-bb38-93ab258b009a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:32.517872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a9c148-fd7a-44b2-bb38-93ab258b009a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:32.525995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba390b57-ae70-421d-aad7-0161b2931a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:32.526813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba390b57-ae70-421d-aad7-0161b2931a00 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:32.527251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba390b57-ae70-421d-aad7-0161b2931a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:32.527872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba390b57-ae70-421d-aad7-0161b2931a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:32.541991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba390b57-ae70-421d-aad7-0161b2931a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:32.547246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba390b57-ae70-421d-aad7-0161b2931a00 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:32.565272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1064/2155] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:33.779599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f151c-2b85-4569-a538-0909a785b735 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:33.781864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dc0f151c-2b85-4569-a538-0909a785b735 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:33.818845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:33.863982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0f151c-2b85-4569-a538-0909a785b735 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:33.868633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:33.869508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:33.869796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:33.870101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] PATH_INFO: `/v3/users/99e31454d9aa47f38f09f016399f9b86` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:33.871359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e31454d9aa47f38f09f016399f9b86)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:33.873283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:33.920849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e31454d9aa47f38f09f016399f9b86'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:33.921483 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e31454d9aa47f38f09f016399f9b86'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:33.930410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e31454d9aa47f38f09f016399f9b86'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:33.947073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e31454d9aa47f38f09f016399f9b86'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:33.947967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a59af1b3-3262-4376-989d-9bb5b395f343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e31454d9aa47f38f09f016399f9b86'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:33.969429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1092/2156] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:33 2022] DELETE /identity/v3/users/99e31454d9aa47f38f09f016399f9b86 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:34.143908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba390b57-ae70-421d-aad7-0161b2931a00 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:34.155403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba390b57-ae70-421d-aad7-0161b2931a00 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:34.211408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:34.212256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.212537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.212809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] PATH_INFO: `/v3/projects/23078d86cd394c749328fb9dd7992919` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.213975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] RBAC: Authorizing `identity:delete_project(project_id=23078d86cd394c749328fb9dd7992919)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:34.215888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:34.252294 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:34.262332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23078d86cd394c749328fb9dd7992919', 'request_id': 'req-936669be-b84f-4a36-96cb-0ad0cb00488b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:34.269922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23078d86cd394c749328fb9dd7992919', 'request_id': 'req-936669be-b84f-4a36-96cb-0ad0cb00488b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:34.275542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23078d86cd394c749328fb9dd7992919', 'request_id': 'req-936669be-b84f-4a36-96cb-0ad0cb00488b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:34.286342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] The token cache is being invalidate because project 23078d86cd394c749328fb9dd7992919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:34.286630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:34.287272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-936669be-b84f-4a36-96cb-0ad0cb00488b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:34.288803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1065/2157] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:34 2022] DELETE /identity/v3/projects/23078d86cd394c749328fb9dd7992919 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:34.293685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94726a3a-2561-4185-856e-69fc868f3f03 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.293981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94726a3a-2561-4185-856e-69fc868f3f03 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.294294 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94726a3a-2561-4185-856e-69fc868f3f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.311816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-94726a3a-2561-4185-856e-69fc868f3f03 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:34.313014 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-94726a3a-2561-4185-856e-69fc868f3f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:34.370338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1093/2158] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:34.378696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b2556-679b-4bc9-8560-2d3a74c81d90 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.379067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b2556-679b-4bc9-8560-2d3a74c81d90 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.379403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74b2556-679b-4bc9-8560-2d3a74c81d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.398775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e74b2556-679b-4bc9-8560-2d3a74c81d90 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:34.400006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e74b2556-679b-4bc9-8560-2d3a74c81d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:34.408245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1066/2159] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:34.412504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94726a3a-2561-4185-856e-69fc868f3f03 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:34.422845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94726a3a-2561-4185-856e-69fc868f3f03 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:34.428698 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee739d5-8055-4c93-8ad5-c0cdff848473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:34.429380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee739d5-8055-4c93-8ad5-c0cdff848473 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.429667 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee739d5-8055-4c93-8ad5-c0cdff848473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.429947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee739d5-8055-4c93-8ad5-c0cdff848473 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.430731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee739d5-8055-4c93-8ad5-c0cdff848473 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:34.431045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee739d5-8055-4c93-8ad5-c0cdff848473 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:34.432659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee739d5-8055-4c93-8ad5-c0cdff848473 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:34.445198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1094/2160] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:06:34.449369 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b2556-679b-4bc9-8560-2d3a74c81d90 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:34.461306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74b2556-679b-4bc9-8560-2d3a74c81d90 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:34.465923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e2a58-6cb5-4834-9559-402fffff09d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:34.466610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e2a58-6cb5-4834-9559-402fffff09d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.466909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e2a58-6cb5-4834-9559-402fffff09d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.467165 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e2a58-6cb5-4834-9559-402fffff09d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.467859 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e2a58-6cb5-4834-9559-402fffff09d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:34.469784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e2a58-6cb5-4834-9559-402fffff09d0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:34.494013 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1067/2161] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:34 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:34.498719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee739d5-8055-4c93-8ad5-c0cdff848473 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:34.509285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee739d5-8055-4c93-8ad5-c0cdff848473 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:34.541641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:34.570295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bee55-2d10-4ce1-b0cf-fe7fd5d1c547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:34.570959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bee55-2d10-4ce1-b0cf-fe7fd5d1c547 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.571238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bee55-2d10-4ce1-b0cf-fe7fd5d1c547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.571511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bee55-2d10-4ce1-b0cf-fe7fd5d1c547 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.572197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bee55-2d10-4ce1-b0cf-fe7fd5d1c547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:34.574084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bee55-2d10-4ce1-b0cf-fe7fd5d1c547 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:34.593305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1095/2162] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:34 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:34.597361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e2a58-6cb5-4834-9559-402fffff09d0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:34.607469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e2a58-6cb5-4834-9559-402fffff09d0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:34.612486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82317a22-0211-4852-85f8-dc503119424e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:34.613138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82317a22-0211-4852-85f8-dc503119424e admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.613418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82317a22-0211-4852-85f8-dc503119424e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.613691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82317a22-0211-4852-85f8-dc503119424e admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.614445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82317a22-0211-4852-85f8-dc503119424e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:34.616019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82317a22-0211-4852-85f8-dc503119424e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:34.627257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1068/2163] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:34.631447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bee55-2d10-4ce1-b0cf-fe7fd5d1c547 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:34.641111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bee55-2d10-4ce1-b0cf-fe7fd5d1c547 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:34.645615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc189cc-af59-44c4-8007-33377ccb03b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:34.646296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc189cc-af59-44c4-8007-33377ccb03b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.646580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc189cc-af59-44c4-8007-33377ccb03b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.646856 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc189cc-af59-44c4-8007-33377ccb03b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.647574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc189cc-af59-44c4-8007-33377ccb03b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:34.649143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc189cc-af59-44c4-8007-33377ccb03b4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:34.659134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1096/2164] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:34.662977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82317a22-0211-4852-85f8-dc503119424e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:34.674017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82317a22-0211-4852-85f8-dc503119424e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:34.679019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec0a33-7566-449d-90a3-d2c88fe12035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:34.680071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec0a33-7566-449d-90a3-d2c88fe12035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.680363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec0a33-7566-449d-90a3-d2c88fe12035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.680680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec0a33-7566-449d-90a3-d2c88fe12035 admin admin] PATH_INFO: `/v3/projects/c3bb6f4e883b42d597f734463d219f21/users/b700a67b0ba241bca89b74f13c50cc4a/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.695995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec0a33-7566-449d-90a3-d2c88fe12035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3bb6f4e883b42d597f734463d219f21, user_id=b700a67b0ba241bca89b74f13c50cc4a, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:34.698696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec0a33-7566-449d-90a3-d2c88fe12035 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:34.711801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1069/2165] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:34 2022] PUT /identity/v3/projects/c3bb6f4e883b42d597f734463d219f21/users/b700a67b0ba241bca89b74f13c50cc4a/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:34.722335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9a927-f4c7-4e33-81f9-b8dc3bee7a3e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.722631 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9a927-f4c7-4e33-81f9-b8dc3bee7a3e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.722910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9a927-f4c7-4e33-81f9-b8dc3bee7a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.750224 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a1a9a927-f4c7-4e33-81f9-b8dc3bee7a3e None None] MFA Rules not processed for user `b700a67b0ba241bca89b74f13c50cc4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:34.750978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a1a9a927-f4c7-4e33-81f9-b8dc3bee7a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:34.814677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1097/2166] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:34.825889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec0a33-7566-449d-90a3-d2c88fe12035 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:34.827615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c0ec0a33-7566-449d-90a3-d2c88fe12035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:34.906673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec0a33-7566-449d-90a3-d2c88fe12035 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:34.912141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b76fd0-e139-4748-8107-b5dda1da7c24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:34.912787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b76fd0-e139-4748-8107-b5dda1da7c24 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:34.913068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b76fd0-e139-4748-8107-b5dda1da7c24 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:34.913341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b76fd0-e139-4748-8107-b5dda1da7c24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:34.921562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b76fd0-e139-4748-8107-b5dda1da7c24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:34.923159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b76fd0-e139-4748-8107-b5dda1da7c24 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:34.931811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1070/2167] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:38.314719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9a927-f4c7-4e33-81f9-b8dc3bee7a3e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:38.316862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a1a9a927-f4c7-4e33-81f9-b8dc3bee7a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:38.397602 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9a927-f4c7-4e33-81f9-b8dc3bee7a3e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:38.402350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:38.403255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:38.403539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:38.403815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] PATH_INFO: `/v3/users/84b99364e5fb439dbfbce21eb9bac786` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:38.405596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b99364e5fb439dbfbce21eb9bac786)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:38.407540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:38.464955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:38.469052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b99364e5fb439dbfbce21eb9bac786'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:38.469951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b99364e5fb439dbfbce21eb9bac786'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:38.482649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b99364e5fb439dbfbce21eb9bac786'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:38.492281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b99364e5fb439dbfbce21eb9bac786'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:38.492948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b99364e5fb439dbfbce21eb9bac786'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:38.519753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1098/2168] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:38 2022] DELETE /identity/v3/users/84b99364e5fb439dbfbce21eb9bac786 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:38.524200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d82a3-040e-42b4-9b38-87c6070bccaf None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:38.524698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d82a3-040e-42b4-9b38-87c6070bccaf None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:38.524779 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34d82a3-040e-42b4-9b38-87c6070bccaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:38.548359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c34d82a3-040e-42b4-9b38-87c6070bccaf None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:38.549674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c34d82a3-040e-42b4-9b38-87c6070bccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:38.601646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:38.640199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1071/2169] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:38.655581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:38.661253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:38.666831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d82a3-040e-42b4-9b38-87c6070bccaf None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:38.672147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c34d82a3-040e-42b4-9b38-87c6070bccaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:38.689332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34d82a3-040e-42b4-9b38-87c6070bccaf None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:38.696953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:38.697829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:38.698140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:38.698416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] PATH_INFO: `/v3/users/b700a67b0ba241bca89b74f13c50cc4a` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:38.699629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b700a67b0ba241bca89b74f13c50cc4a)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:38.702219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:38.742117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b108e964-360a-4e63-b0bd-3a46cb57b547 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:38.746276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d5fd29-10d7-4968-87ef-fc8473c2b584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:38.746952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5fd29-10d7-4968-87ef-fc8473c2b584 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:38.747239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5fd29-10d7-4968-87ef-fc8473c2b584 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:38.747616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d5fd29-10d7-4968-87ef-fc8473c2b584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:38.758611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5fd29-10d7-4968-87ef-fc8473c2b584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:38.760785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d5fd29-10d7-4968-87ef-fc8473c2b584 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:38.761779 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b700a67b0ba241bca89b74f13c50cc4a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:38.763590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b700a67b0ba241bca89b74f13c50cc4a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:38.764952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b700a67b0ba241bca89b74f13c50cc4a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:38.772551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1099/2170] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:38.792808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b700a67b0ba241bca89b74f13c50cc4a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:38.803010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b700a67b0ba241bca89b74f13c50cc4a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:38.822697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1072/2171] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:38 2022] DELETE /identity/v3/users/b700a67b0ba241bca89b74f13c50cc4a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:38.827611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173ae9-1626-4b2c-b991-47b7d72b6f38 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:38.827938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173ae9-1626-4b2c-b991-47b7d72b6f38 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:38.828218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff173ae9-1626-4b2c-b991-47b7d72b6f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:38.846824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ff173ae9-1626-4b2c-b991-47b7d72b6f38 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:38.848000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ff173ae9-1626-4b2c-b991-47b7d72b6f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:38.929876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1100/2172] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:38.980053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:38.983441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:39.045188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c666ea50-fb7d-47e5-933f-027fd7d70981 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:39.049642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dae75-d82a-446a-8d8b-c75f5a5d5e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:39.050333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dae75-d82a-446a-8d8b-c75f5a5d5e24 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:39.050648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dae75-d82a-446a-8d8b-c75f5a5d5e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:39.050978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dae75-d82a-446a-8d8b-c75f5a5d5e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:39.061053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dae75-d82a-446a-8d8b-c75f5a5d5e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:39.062675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dae75-d82a-446a-8d8b-c75f5a5d5e24 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:39.073866 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1073/2173] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:39.390499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff173ae9-1626-4b2c-b991-47b7d72b6f38 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:39.393488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ff173ae9-1626-4b2c-b991-47b7d72b6f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:39.407347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff173ae9-1626-4b2c-b991-47b7d72b6f38 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:39.412894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:39.413845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:39.414366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:39.414801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] PATH_INFO: `/v3/projects/841135cad3e24ddd92df811779f6595a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:39.424538 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=841135cad3e24ddd92df811779f6595a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:39.433002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:39.506718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '841135cad3e24ddd92df811779f6595a', 'request_id': 'req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.512898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '841135cad3e24ddd92df811779f6595a', 'request_id': 'req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.516493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:39.522426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '841135cad3e24ddd92df811779f6595a', 'request_id': 'req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.537317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] The token cache is being invalidate because project 841135cad3e24ddd92df811779f6595a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:39.537744 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.538499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.554489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1101/2174] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:39 2022] DELETE /identity/v3/projects/841135cad3e24ddd92df811779f6595a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:39.694426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dae75-d82a-446a-8d8b-c75f5a5d5e24 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:39.695946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-444dae75-d82a-446a-8d8b-c75f5a5d5e24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:39.735317 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dae75-d82a-446a-8d8b-c75f5a5d5e24 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:39.808608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:39.809569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:39.809928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:39.810304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] PATH_INFO: `/v3/projects/c3bb6f4e883b42d597f734463d219f21` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:39.811869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3bb6f4e883b42d597f734463d219f21)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:39.813946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:39.815536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:39.817769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:39.821558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:39.835126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0a4c0-dd26-42d0-8ae3-5c7011d6ee4e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:39.840127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:39.841113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:39.841468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:39.842460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] PATH_INFO: `/v3/users/a4603fc7a991497e8757dca042e8df23` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:39.843748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4603fc7a991497e8757dca042e8df23)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:39.845734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:39.875801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3bb6f4e883b42d597f734463d219f21', 'request_id': 'req-07fda998-b8d4-4d08-925f-1a3b168abf31'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.883855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3bb6f4e883b42d597f734463d219f21', 'request_id': 'req-07fda998-b8d4-4d08-925f-1a3b168abf31'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.892974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3bb6f4e883b42d597f734463d219f21', 'request_id': 'req-07fda998-b8d4-4d08-925f-1a3b168abf31'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.905927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4603fc7a991497e8757dca042e8df23'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.906622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4603fc7a991497e8757dca042e8df23'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.906925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] The token cache is being invalidate because project c3bb6f4e883b42d597f734463d219f21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:39.907216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.907902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-07fda998-b8d4-4d08-925f-1a3b168abf31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.909677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1074/2175] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:39 2022] DELETE /identity/v3/projects/c3bb6f4e883b42d597f734463d219f21 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:39.914199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a8ee5-7077-464d-8de2-c720c2dc6922 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:39.914490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a8ee5-7077-464d-8de2-c720c2dc6922 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:39.914769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a8ee5-7077-464d-8de2-c720c2dc6922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:39.920179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4603fc7a991497e8757dca042e8df23'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.932047 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4603fc7a991497e8757dca042e8df23'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.933454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-9a83fd6b-2d07-449f-90fb-291c77e56b9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4603fc7a991497e8757dca042e8df23'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:39.934568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ba5a8ee5-7077-464d-8de2-c720c2dc6922 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:39.937014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ba5a8ee5-7077-464d-8de2-c720c2dc6922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:39.957834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1102/2176] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:39 2022] DELETE /identity/v3/users/a4603fc7a991497e8757dca042e8df23 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:39.965590 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc196d8-7b1e-4bb4-975c-3039e86093aa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:39.967665 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc196d8-7b1e-4bb4-975c-3039e86093aa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:39.968063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc196d8-7b1e-4bb4-975c-3039e86093aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:39.990318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-2cc196d8-7b1e-4bb4-975c-3039e86093aa None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:39.991648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2cc196d8-7b1e-4bb4-975c-3039e86093aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:40.023404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1075/2177] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:40.031517 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aafbc-1fc2-4d6f-b854-29cdfc0e79d3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.031810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aafbc-1fc2-4d6f-b854-29cdfc0e79d3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.032090 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6aafbc-1fc2-4d6f-b854-29cdfc0e79d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.052203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7d6aafbc-1fc2-4d6f-b854-29cdfc0e79d3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:40.054945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7d6aafbc-1fc2-4d6f-b854-29cdfc0e79d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:40.062474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1103/2178] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:40.071934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1076/2179] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:40.072303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc196d8-7b1e-4bb4-975c-3039e86093aa None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.074116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2cc196d8-7b1e-4bb4-975c-3039e86093aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:40.075969 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6aafbc-1fc2-4d6f-b854-29cdfc0e79d3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.087850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6aafbc-1fc2-4d6f-b854-29cdfc0e79d3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.094411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b61a9-3bc4-432e-a300-a56cba5fc049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.094866 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b61a9-3bc4-432e-a300-a56cba5fc049 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.095145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b61a9-3bc4-432e-a300-a56cba5fc049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.095456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b61a9-3bc4-432e-a300-a56cba5fc049 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.096134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b61a9-3bc4-432e-a300-a56cba5fc049 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:40.096438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b61a9-3bc4-432e-a300-a56cba5fc049 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.098024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b61a9-3bc4-432e-a300-a56cba5fc049 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.110655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1077/2180] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:06:40.114940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b61a9-3bc4-432e-a300-a56cba5fc049 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.125582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b61a9-3bc4-432e-a300-a56cba5fc049 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.130478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f113a9-3449-4a1f-b6c1-86a1c4486152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.131120 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f113a9-3449-4a1f-b6c1-86a1c4486152 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.131407 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f113a9-3449-4a1f-b6c1-86a1c4486152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.131680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f113a9-3449-4a1f-b6c1-86a1c4486152 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.132370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f113a9-3449-4a1f-b6c1-86a1c4486152 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.134281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f113a9-3449-4a1f-b6c1-86a1c4486152 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.144173 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc196d8-7b1e-4bb4-975c-3039e86093aa None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.148603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49508ce2-578f-4f83-b044-63bf4474f93e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.149687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49508ce2-578f-4f83-b044-63bf4474f93e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.149687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49508ce2-578f-4f83-b044-63bf4474f93e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.150733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49508ce2-578f-4f83-b044-63bf4474f93e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.163464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49508ce2-578f-4f83-b044-63bf4474f93e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.164699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1078/2181] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:40 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:40.165390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49508ce2-578f-4f83-b044-63bf4474f93e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.169027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f113a9-3449-4a1f-b6c1-86a1c4486152 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.177348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1104/2182] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:40.228748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:40.247458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f113a9-3449-4a1f-b6c1-86a1c4486152 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.254494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec63753-109b-4619-b6ca-b14f84ed3e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.255755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec63753-109b-4619-b6ca-b14f84ed3e8f admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.255755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec63753-109b-4619-b6ca-b14f84ed3e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.255755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec63753-109b-4619-b6ca-b14f84ed3e8f admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.256656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec63753-109b-4619-b6ca-b14f84ed3e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.258723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec63753-109b-4619-b6ca-b14f84ed3e8f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.282998 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1079/2183] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:40 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:40.286966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49508ce2-578f-4f83-b044-63bf4474f93e service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.295463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:40.298920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49508ce2-578f-4f83-b044-63bf4474f93e service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.305473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c510d04-ae66-476a-baf5-b9b70dfb2665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.305668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c510d04-ae66-476a-baf5-b9b70dfb2665 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.305896 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c510d04-ae66-476a-baf5-b9b70dfb2665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.306149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c510d04-ae66-476a-baf5-b9b70dfb2665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.306887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c510d04-ae66-476a-baf5-b9b70dfb2665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.308648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c510d04-ae66-476a-baf5-b9b70dfb2665 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.336317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1105/2184] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:40.339808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec63753-109b-4619-b6ca-b14f84ed3e8f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.352156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec63753-109b-4619-b6ca-b14f84ed3e8f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.356904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ebe095-cc27-4f2b-a380-fd904b1df10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.357601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebe095-cc27-4f2b-a380-fd904b1df10b admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.357884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebe095-cc27-4f2b-a380-fd904b1df10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.358340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ebe095-cc27-4f2b-a380-fd904b1df10b admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.359074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebe095-cc27-4f2b-a380-fd904b1df10b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.362669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ebe095-cc27-4f2b-a380-fd904b1df10b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.377591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1080/2185] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:40.381813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c510d04-ae66-476a-baf5-b9b70dfb2665 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.392400 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c510d04-ae66-476a-baf5-b9b70dfb2665 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.397077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2074d03-7bea-413b-a6ee-4ed6b12030c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.397977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2074d03-7bea-413b-a6ee-4ed6b12030c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.398303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2074d03-7bea-413b-a6ee-4ed6b12030c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.398576 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2074d03-7bea-413b-a6ee-4ed6b12030c5 admin admin] PATH_INFO: `/v3/projects/310662670edb4aa6b7763c4f5b905ca1/users/1a22763ce5bb426db0ba79bbb4a7636d/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.414016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2074d03-7bea-413b-a6ee-4ed6b12030c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=310662670edb4aa6b7763c4f5b905ca1, user_id=1a22763ce5bb426db0ba79bbb4a7636d, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.416842 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2074d03-7bea-413b-a6ee-4ed6b12030c5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.441121 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1106/2186] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:40 2022] PUT /identity/v3/projects/310662670edb4aa6b7763c4f5b905ca1/users/1a22763ce5bb426db0ba79bbb4a7636d/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:40.444525 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f12c6-a65b-4e99-aeb4-36a9a801684b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.444844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f12c6-a65b-4e99-aeb4-36a9a801684b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.445174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2f12c6-a65b-4e99-aeb4-36a9a801684b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.464418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2074d03-7bea-413b-a6ee-4ed6b12030c5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.467093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b2074d03-7bea-413b-a6ee-4ed6b12030c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:40.515442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6f2f12c6-a65b-4e99-aeb4-36a9a801684b None None] MFA Rules not processed for user `1a22763ce5bb426db0ba79bbb4a7636d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:40.516783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6f2f12c6-a65b-4e99-aeb4-36a9a801684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:40.557348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2074d03-7bea-413b-a6ee-4ed6b12030c5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.564430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.565768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.566162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.566498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] PATH_INFO: `/v3/projects/1e93a6fa3aa94857bde480c34b0051aa` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.567772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e93a6fa3aa94857bde480c34b0051aa)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.570198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.589572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1081/2187] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:40.594241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f12c6-a65b-4e99-aeb4-36a9a801684b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.604465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2f12c6-a65b-4e99-aeb4-36a9a801684b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.612004 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87387490-93a9-42c0-aae9-c1a9de7290e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.612746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87387490-93a9-42c0-aae9-c1a9de7290e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.613323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87387490-93a9-42c0-aae9-c1a9de7290e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.613564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87387490-93a9-42c0-aae9-c1a9de7290e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.614897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87387490-93a9-42c0-aae9-c1a9de7290e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.617382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87387490-93a9-42c0-aae9-c1a9de7290e0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.634330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e93a6fa3aa94857bde480c34b0051aa', 'request_id': 'req-ed7cdbbd-148a-4e91-9b78-2697389639d8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:40.641721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e93a6fa3aa94857bde480c34b0051aa', 'request_id': 'req-ed7cdbbd-148a-4e91-9b78-2697389639d8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:40.649147 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e93a6fa3aa94857bde480c34b0051aa', 'request_id': 'req-ed7cdbbd-148a-4e91-9b78-2697389639d8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:40.649990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1082/2188] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:40 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:40.654214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87387490-93a9-42c0-aae9-c1a9de7290e0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.665613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] The token cache is being invalidate because project 1e93a6fa3aa94857bde480c34b0051aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:40.665959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:40.666664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:40.668427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1107/2189] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:40 2022] DELETE /identity/v3/projects/1e93a6fa3aa94857bde480c34b0051aa => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:40.672976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.675569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:40.741917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87387490-93a9-42c0-aae9-c1a9de7290e0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.747000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67a82c-0a9f-42ef-b635-440e3ccd491b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.749067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67a82c-0a9f-42ef-b635-440e3ccd491b admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.749572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67a82c-0a9f-42ef-b635-440e3ccd491b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.749927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67a82c-0a9f-42ef-b635-440e3ccd491b admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.750827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67a82c-0a9f-42ef-b635-440e3ccd491b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.751356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7cdbbd-148a-4e91-9b78-2697389639d8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.752926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67a82c-0a9f-42ef-b635-440e3ccd491b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.757726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.758655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.758937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.759208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] PATH_INFO: `/v3/users/1b8a8dce306b49278589b8fef1973650` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.761076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8a8dce306b49278589b8fef1973650)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.763073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.764982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:40.780876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1083/2190] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:40 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:40.784315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67a82c-0a9f-42ef-b635-440e3ccd491b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.786457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8b67a82c-0a9f-42ef-b635-440e3ccd491b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:40.801340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67a82c-0a9f-42ef-b635-440e3ccd491b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.806749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc70fea-24f6-4017-b579-4307310d99d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.807453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc70fea-24f6-4017-b579-4307310d99d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.807678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc70fea-24f6-4017-b579-4307310d99d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.807980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc70fea-24f6-4017-b579-4307310d99d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.808710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc70fea-24f6-4017-b579-4307310d99d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.810334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc70fea-24f6-4017-b579-4307310d99d2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.822089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1084/2191] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:40.826387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc70fea-24f6-4017-b579-4307310d99d2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.829987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8a8dce306b49278589b8fef1973650'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:40.830939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8a8dce306b49278589b8fef1973650'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:40.836695 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc70fea-24f6-4017-b579-4307310d99d2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:40.838578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8a8dce306b49278589b8fef1973650'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:40.841184 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a16294-8390-4bf0-bf6a-281cdaaf3b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:40.842150 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16294-8390-4bf0-bf6a-281cdaaf3b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:40.842437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16294-8390-4bf0-bf6a-281cdaaf3b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:40.842711 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16294-8390-4bf0-bf6a-281cdaaf3b48 admin admin] PATH_INFO: `/v3/projects/406947a875b94e649f36ae2f6c31e1aa/users/807eb98d10cf4dbf835f305b15f35856/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:40.854616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8a8dce306b49278589b8fef1973650'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:40.855867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:40.856738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8a8dce306b49278589b8fef1973650'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:40.862331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a16294-8390-4bf0-bf6a-281cdaaf3b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=406947a875b94e649f36ae2f6c31e1aa, user_id=807eb98d10cf4dbf835f305b15f35856, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:40.865914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a16294-8390-4bf0-bf6a-281cdaaf3b48 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:40.882179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1085/2192] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:40 2022] PUT /identity/v3/projects/406947a875b94e649f36ae2f6c31e1aa/users/807eb98d10cf4dbf835f305b15f35856/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:40.885864 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a16294-8390-4bf0-bf6a-281cdaaf3b48 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:40.886619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1108/2193] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:40 2022] DELETE /identity/v3/users/1b8a8dce306b49278589b8fef1973650 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:40.887857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c8a16294-8390-4bf0-bf6a-281cdaaf3b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:40.956053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a16294-8390-4bf0-bf6a-281cdaaf3b48 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:41.026465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59348c8-bc1f-426b-9c31-666ccf9ddac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:41.027134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59348c8-bc1f-426b-9c31-666ccf9ddac7 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:41.027473 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59348c8-bc1f-426b-9c31-666ccf9ddac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:41.027690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59348c8-bc1f-426b-9c31-666ccf9ddac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:41.028429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59348c8-bc1f-426b-9c31-666ccf9ddac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:41.030653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59348c8-bc1f-426b-9c31-666ccf9ddac7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:41.046234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1086/2194] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:41.048445 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:41.059441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba33062-9c43-4280-8bba-bcbdce02ac85 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:41.064335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e045-441a-4c3f-90ab-86a84998f984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:41.065339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e045-441a-4c3f-90ab-86a84998f984 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:41.065723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e045-441a-4c3f-90ab-86a84998f984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:41.066146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2613e045-441a-4c3f-90ab-86a84998f984 admin admin] PATH_INFO: `/v3/projects/406947a875b94e649f36ae2f6c31e1aa/users/807eb98d10cf4dbf835f305b15f35856/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:41.068283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e045-441a-4c3f-90ab-86a84998f984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=406947a875b94e649f36ae2f6c31e1aa, user_id=807eb98d10cf4dbf835f305b15f35856, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:41.070879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2613e045-441a-4c3f-90ab-86a84998f984 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:41.084932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1109/2195] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:41 2022] PUT /identity/v3/projects/406947a875b94e649f36ae2f6c31e1aa/users/807eb98d10cf4dbf835f305b15f35856/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:41.097047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2a92a-264c-4dee-bcf6-81a6b02bbd4f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:41.097338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2a92a-264c-4dee-bcf6-81a6b02bbd4f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:41.097616 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2a92a-264c-4dee-bcf6-81a6b02bbd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:41.127246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-20f2a92a-264c-4dee-bcf6-81a6b02bbd4f None None] MFA Rules not processed for user `807eb98d10cf4dbf835f305b15f35856`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:41.128724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-20f2a92a-264c-4dee-bcf6-81a6b02bbd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:41.179287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e045-441a-4c3f-90ab-86a84998f984 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:41.181075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2613e045-441a-4c3f-90ab-86a84998f984 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:41.222199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1087/2196] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:41 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:41.236773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2a92a-264c-4dee-bcf6-81a6b02bbd4f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:41.238334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-20f2a92a-264c-4dee-bcf6-81a6b02bbd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:41.258966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:41.269899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2613e045-441a-4c3f-90ab-86a84998f984 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:41.275968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:41.276839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:41.277125 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:41.277408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] PATH_INFO: `/v3/projects/243bb5851f5849fdab6410f4c114f4e2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:41.279218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] RBAC: Authorizing `identity:delete_project(project_id=243bb5851f5849fdab6410f4c114f4e2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:41.281712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:41.304342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2a92a-264c-4dee-bcf6-81a6b02bbd4f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:41.308151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e5d9c-d577-4c97-9233-9f31f1ee2403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:41.308886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e5d9c-d577-4c97-9233-9f31f1ee2403 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:41.309270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e5d9c-d577-4c97-9233-9f31f1ee2403 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:41.309542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e5d9c-d577-4c97-9233-9f31f1ee2403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:41.321296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e5d9c-d577-4c97-9233-9f31f1ee2403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:41.322967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e5d9c-d577-4c97-9233-9f31f1ee2403 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:41.333175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1088/2197] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:06:41.338253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '243bb5851f5849fdab6410f4c114f4e2', 'request_id': 'req-fc7c2754-b479-4571-8a97-6a2721682440'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.345813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '243bb5851f5849fdab6410f4c114f4e2', 'request_id': 'req-fc7c2754-b479-4571-8a97-6a2721682440'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.355444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '243bb5851f5849fdab6410f4c114f4e2', 'request_id': 'req-fc7c2754-b479-4571-8a97-6a2721682440'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.358524 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:41.371937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] The token cache is being invalidate because project 243bb5851f5849fdab6410f4c114f4e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:41.372521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.373715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.375476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1110/2198] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:41 2022] DELETE /identity/v3/projects/243bb5851f5849fdab6410f4c114f4e2 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:41.379733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e5d9c-d577-4c97-9233-9f31f1ee2403 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:41.393540 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e5d9c-d577-4c97-9233-9f31f1ee2403 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:41.469467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:41.470382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:41.470673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:41.471095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] PATH_INFO: `/v3/users/770f4f71bbff4f7fa1209a3a4f7e2836` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:41.472337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770f4f71bbff4f7fa1209a3a4f7e2836)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:41.474477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:41.529453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770f4f71bbff4f7fa1209a3a4f7e2836'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.533634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770f4f71bbff4f7fa1209a3a4f7e2836'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.533634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770f4f71bbff4f7fa1209a3a4f7e2836'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.550259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770f4f71bbff4f7fa1209a3a4f7e2836'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.564209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770f4f71bbff4f7fa1209a3a4f7e2836'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.583207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1089/2199] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:41 2022] DELETE /identity/v3/users/770f4f71bbff4f7fa1209a3a4f7e2836 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:41.841573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:41.843669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:41.861194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c2754-b479-4571-8a97-6a2721682440 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:41.939575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:41.941128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:41.942025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:41.943039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:41.943391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:41.943753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] PATH_INFO: `/v3/users/1a22763ce5bb426db0ba79bbb4a7636d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:41.945123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a22763ce5bb426db0ba79bbb4a7636d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:41.948378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:41.956295 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d992674b-e226-469a-8265-a7147c3f3557 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:41.961630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:41.962720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:41.963015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:41.963293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] PATH_INFO: `/v3/projects/3b510776db304275838a44767e366ead` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:41.964513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b510776db304275838a44767e366ead)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:41.966480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:41.999089 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a22763ce5bb426db0ba79bbb4a7636d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:41.999930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a22763ce5bb426db0ba79bbb4a7636d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.006842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:42.008905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a22763ce5bb426db0ba79bbb4a7636d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.016885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a22763ce5bb426db0ba79bbb4a7636d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.017505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-05083349-e883-4d5c-a883-1ee6d889d393 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a22763ce5bb426db0ba79bbb4a7636d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.019233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b510776db304275838a44767e366ead', 'request_id': 'req-15645aac-91a5-4c3c-959f-1afb3d903b24'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.028088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b510776db304275838a44767e366ead', 'request_id': 'req-15645aac-91a5-4c3c-959f-1afb3d903b24'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.034604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b510776db304275838a44767e366ead', 'request_id': 'req-15645aac-91a5-4c3c-959f-1afb3d903b24'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.038904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1111/2200] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:41 2022] DELETE /identity/v3/users/1a22763ce5bb426db0ba79bbb4a7636d => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:42.043343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a93c4-37c6-40bf-9a79-fe26a314c65a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.043634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a93c4-37c6-40bf-9a79-fe26a314c65a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.043931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a93c4-37c6-40bf-9a79-fe26a314c65a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.048608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] The token cache is being invalidate because project 3b510776db304275838a44767e366ead was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:42.048907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.049568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-15645aac-91a5-4c3c-959f-1afb3d903b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.051133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1090/2201] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:41 2022] DELETE /identity/v3/projects/3b510776db304275838a44767e366ead => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:42.055807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5639a69-53ee-4885-b43f-eb850e4c1e0d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.056095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5639a69-53ee-4885-b43f-eb850e4c1e0d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.056370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5639a69-53ee-4885-b43f-eb850e4c1e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.060793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4b1a93c4-37c6-40bf-9a79-fe26a314c65a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:42.062424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4b1a93c4-37c6-40bf-9a79-fe26a314c65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:42.075536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c5639a69-53ee-4885-b43f-eb850e4c1e0d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:42.077539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c5639a69-53ee-4885-b43f-eb850e4c1e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:42.129080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1112/2202] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:42.139181 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a93c4-37c6-40bf-9a79-fe26a314c65a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:42.139642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1091/2203] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:42.143256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4b1a93c4-37c6-40bf-9a79-fe26a314c65a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:42.147043 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99575d-2959-40e5-ac81-9a47b77edb57 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.147339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99575d-2959-40e5-ac81-9a47b77edb57 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.147620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd99575d-2959-40e5-ac81-9a47b77edb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.162457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:42.166189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cd99575d-2959-40e5-ac81-9a47b77edb57 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:42.167454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cd99575d-2959-40e5-ac81-9a47b77edb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:42.175839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1092/2204] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:42.179987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99575d-2959-40e5-ac81-9a47b77edb57 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:42.188905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd99575d-2959-40e5-ac81-9a47b77edb57 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:42.193569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e023e1-7ccb-463e-be16-83c7c0ebcc02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:42.194243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e023e1-7ccb-463e-be16-83c7c0ebcc02 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.194532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e023e1-7ccb-463e-be16-83c7c0ebcc02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.194813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e023e1-7ccb-463e-be16-83c7c0ebcc02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.195547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e023e1-7ccb-463e-be16-83c7c0ebcc02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:42.195860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e023e1-7ccb-463e-be16-83c7c0ebcc02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:42.197458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e023e1-7ccb-463e-be16-83c7c0ebcc02 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:42.203766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a93c4-37c6-40bf-9a79-fe26a314c65a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:42.207423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79c52-6fa0-4783-9d61-a86b945960c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:42.208068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79c52-6fa0-4783-9d61-a86b945960c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.208354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79c52-6fa0-4783-9d61-a86b945960c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.208633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d79c52-6fa0-4783-9d61-a86b945960c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.209581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1093/2205] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:06:42.214178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e023e1-7ccb-463e-be16-83c7c0ebcc02 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:42.218566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d79c52-6fa0-4783-9d61-a86b945960c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:42.219993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d79c52-6fa0-4783-9d61-a86b945960c6 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:42.224175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e023e1-7ccb-463e-be16-83c7c0ebcc02 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:42.228862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c436f-39d7-485f-add5-5ffb57e42cbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:42.229519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c436f-39d7-485f-add5-5ffb57e42cbb admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.229807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c436f-39d7-485f-add5-5ffb57e42cbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.230023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1113/2206] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:06:42.234162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217c436f-39d7-485f-add5-5ffb57e42cbb admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.234923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c436f-39d7-485f-add5-5ffb57e42cbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:42.237097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217c436f-39d7-485f-add5-5ffb57e42cbb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:42.269480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1094/2207] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:42 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:42.270789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79c52-6fa0-4783-9d61-a86b945960c6 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:42.281437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d79c52-6fa0-4783-9d61-a86b945960c6 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:42.340140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bbd87-7798-49e3-ba4f-6b6548f4c6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:42.340809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbd87-7798-49e3-ba4f-6b6548f4c6de admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.341092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbd87-7798-49e3-ba4f-6b6548f4c6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.341367 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39bbd87-7798-49e3-ba4f-6b6548f4c6de admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.342750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bbd87-7798-49e3-ba4f-6b6548f4c6de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:42.344654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39bbd87-7798-49e3-ba4f-6b6548f4c6de admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:42.367898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1114/2208] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:42 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:42.369199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c436f-39d7-485f-add5-5ffb57e42cbb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:42.392801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217c436f-39d7-485f-add5-5ffb57e42cbb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:42.398073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f19cb0-73e3-4b61-9fd2-adf3dac3fcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:42.398811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f19cb0-73e3-4b61-9fd2-adf3dac3fcf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.399095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f19cb0-73e3-4b61-9fd2-adf3dac3fcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.399363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f19cb0-73e3-4b61-9fd2-adf3dac3fcf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.400093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f19cb0-73e3-4b61-9fd2-adf3dac3fcf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:42.401677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f19cb0-73e3-4b61-9fd2-adf3dac3fcf9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:42.413067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1095/2209] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:42.417041 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bbd87-7798-49e3-ba4f-6b6548f4c6de admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:42.428337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39bbd87-7798-49e3-ba4f-6b6548f4c6de admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:42.433109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c933d7-8237-461b-83ea-4d53ed1d04a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:42.433774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c933d7-8237-461b-83ea-4d53ed1d04a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.434112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c933d7-8237-461b-83ea-4d53ed1d04a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.434404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c933d7-8237-461b-83ea-4d53ed1d04a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.435142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c933d7-8237-461b-83ea-4d53ed1d04a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:42.436778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c933d7-8237-461b-83ea-4d53ed1d04a0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:42.447450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1115/2210] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:42 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:42.451810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f19cb0-73e3-4b61-9fd2-adf3dac3fcf9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:42.461892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f19cb0-73e3-4b61-9fd2-adf3dac3fcf9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:42.466582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdc9d8-e37e-40d7-8fa2-42cc227709d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:42.468068 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdc9d8-e37e-40d7-8fa2-42cc227709d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.468460 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdc9d8-e37e-40d7-8fa2-42cc227709d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.468745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdc9d8-e37e-40d7-8fa2-42cc227709d6 admin admin] PATH_INFO: `/v3/projects/e77cdf2a35d840848340bce1a1908284/users/ff5253df3ed345bd9adf722fdfee70ad/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.484266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdc9d8-e37e-40d7-8fa2-42cc227709d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77cdf2a35d840848340bce1a1908284, user_id=ff5253df3ed345bd9adf722fdfee70ad, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:42.486952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fdc9d8-e37e-40d7-8fa2-42cc227709d6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:42.501372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1096/2211] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:42 2022] PUT /identity/v3/projects/e77cdf2a35d840848340bce1a1908284/users/ff5253df3ed345bd9adf722fdfee70ad/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:42.517154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c933d7-8237-461b-83ea-4d53ed1d04a0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:42.519302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-67c933d7-8237-461b-83ea-4d53ed1d04a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:42.537478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c933d7-8237-461b-83ea-4d53ed1d04a0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:42.570263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec97ef7-b4a4-45a6-9367-fe474ee600e6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.570658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec97ef7-b4a4-45a6-9367-fe474ee600e6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.571034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec97ef7-b4a4-45a6-9367-fe474ee600e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.599824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cec97ef7-b4a4-45a6-9367-fe474ee600e6 None None] MFA Rules not processed for user `ff5253df3ed345bd9adf722fdfee70ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:42.601026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cec97ef7-b4a4-45a6-9367-fe474ee600e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:42.602808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:42.603802 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.604092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.605506 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] PATH_INFO: `/v3/projects/310662670edb4aa6b7763c4f5b905ca1` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.607219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=310662670edb4aa6b7763c4f5b905ca1)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:42.609706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:42.637240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:42.651853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:42.663388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '310662670edb4aa6b7763c4f5b905ca1', 'request_id': 'req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.670901 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '310662670edb4aa6b7763c4f5b905ca1', 'request_id': 'req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.673018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1097/2212] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:42.679559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '310662670edb4aa6b7763c4f5b905ca1', 'request_id': 'req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.684848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec97ef7-b4a4-45a6-9367-fe474ee600e6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:42.686373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cec97ef7-b4a4-45a6-9367-fe474ee600e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:42.692176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] The token cache is being invalidate because project 310662670edb4aa6b7763c4f5b905ca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:42.692471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.693123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.694733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1116/2213] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:42 2022] DELETE /identity/v3/projects/310662670edb4aa6b7763c4f5b905ca1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:42.698722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:42.700089 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:42.749083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec97ef7-b4a4-45a6-9367-fe474ee600e6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:42.768792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc5fc1-6b32-4abe-ae6a-4cb02f2833f2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:42.775471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:42.776653 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.777107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.777533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] PATH_INFO: `/v3/users/807eb98d10cf4dbf835f305b15f35856` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.782374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] RBAC: Authorizing `identity:delete_user(user_id=807eb98d10cf4dbf835f305b15f35856)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:42.784339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:42.807945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1381534-0ba9-41bb-ae36-b31bfdfb3c91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:42.809182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1381534-0ba9-41bb-ae36-b31bfdfb3c91 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:42.809182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1381534-0ba9-41bb-ae36-b31bfdfb3c91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:42.809402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1381534-0ba9-41bb-ae36-b31bfdfb3c91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:42.812024 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d1381534-0ba9-41bb-ae36-b31bfdfb3c91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:42.843341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '807eb98d10cf4dbf835f305b15f35856'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.844201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '807eb98d10cf4dbf835f305b15f35856'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.851423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '807eb98d10cf4dbf835f305b15f35856'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.860257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '807eb98d10cf4dbf835f305b15f35856'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.862430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '807eb98d10cf4dbf835f305b15f35856'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:42.888018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1117/2214] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:42 2022] DELETE /identity/v3/users/807eb98d10cf4dbf835f305b15f35856 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:42.894457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1381534-0ba9-41bb-ae36-b31bfdfb3c91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:42.895954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1381534-0ba9-41bb-ae36-b31bfdfb3c91 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:42.906445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1098/2215] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:06:43.187057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:43.188523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:43.258547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:43.280990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa71128-b0fc-4ef2-b589-5a3c265ef27a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:43.286204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:43.287118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:43.287398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:43.287679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] PATH_INFO: `/v3/projects/406947a875b94e649f36ae2f6c31e1aa` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:43.289404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=406947a875b94e649f36ae2f6c31e1aa)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:43.292230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:43.348090 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '406947a875b94e649f36ae2f6c31e1aa', 'request_id': 'req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:43.353620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '406947a875b94e649f36ae2f6c31e1aa', 'request_id': 'req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:43.361272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '406947a875b94e649f36ae2f6c31e1aa', 'request_id': 'req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:43.373122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] The token cache is being invalidate because project 406947a875b94e649f36ae2f6c31e1aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:43.373410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:43.374057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-29abd67b-4aa5-46d9-b698-c9c54d10fe1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:43.375660 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1118/2216] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:43 2022] DELETE /identity/v3/projects/406947a875b94e649f36ae2f6c31e1aa => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:43.380955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4ab6a-f589-4083-b7eb-11991e508f9c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:43.381419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4ab6a-f589-4083-b7eb-11991e508f9c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:43.381544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c4ab6a-f589-4083-b7eb-11991e508f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:43.399000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f0c4ab6a-f589-4083-b7eb-11991e508f9c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:43.400257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f0c4ab6a-f589-4083-b7eb-11991e508f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:43.442930 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:43.460611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1099/2217] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:43.468917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6bdc5-945e-43c5-aae0-136cce878ae6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:43.469206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6bdc5-945e-43c5-aae0-136cce878ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:43.469512 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba6bdc5-945e-43c5-aae0-136cce878ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:43.487784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5ba6bdc5-945e-43c5-aae0-136cce878ae6 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:43.488977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5ba6bdc5-945e-43c5-aae0-136cce878ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:43.496737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1119/2218] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:43.501714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4ab6a-f589-4083-b7eb-11991e508f9c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:43.511972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c4ab6a-f589-4083-b7eb-11991e508f9c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:43.517067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31bbff-4f39-4331-9925-b1feda6356a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:43.518153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31bbff-4f39-4331-9925-b1feda6356a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:43.518442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31bbff-4f39-4331-9925-b1feda6356a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:43.518720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f31bbff-4f39-4331-9925-b1feda6356a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:43.519449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31bbff-4f39-4331-9925-b1feda6356a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:43.519757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31bbff-4f39-4331-9925-b1feda6356a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:43.521354 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f31bbff-4f39-4331-9925-b1feda6356a3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:43.535876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1100/2219] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:06:43.539166 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6bdc5-945e-43c5-aae0-136cce878ae6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:43.550958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba6bdc5-945e-43c5-aae0-136cce878ae6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:43.556474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb7349-b444-45b2-9f4c-42bf5df68dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:43.557138 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb7349-b444-45b2-9f4c-42bf5df68dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:43.557423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb7349-b444-45b2-9f4c-42bf5df68dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:43.557765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eb7349-b444-45b2-9f4c-42bf5df68dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:43.558420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb7349-b444-45b2-9f4c-42bf5df68dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:43.560835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eb7349-b444-45b2-9f4c-42bf5df68dd8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:43.588529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1120/2220] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:43 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:43.593947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31bbff-4f39-4331-9925-b1feda6356a3 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:43.604467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f31bbff-4f39-4331-9925-b1feda6356a3 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:43.668600 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0ccaa-3fb7-4d3c-b198-717be0f5e2d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:43.669252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0ccaa-3fb7-4d3c-b198-717be0f5e2d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:43.669590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0ccaa-3fb7-4d3c-b198-717be0f5e2d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:43.669885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca0ccaa-3fb7-4d3c-b198-717be0f5e2d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:43.670707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0ccaa-3fb7-4d3c-b198-717be0f5e2d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:43.673091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca0ccaa-3fb7-4d3c-b198-717be0f5e2d6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:43.697164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1101/2221] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:43 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:43.701739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb7349-b444-45b2-9f4c-42bf5df68dd8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:43.713501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eb7349-b444-45b2-9f4c-42bf5df68dd8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:43.721063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eec4c5-3258-434a-8450-4e9a16afef53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:43.721960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec4c5-3258-434a-8450-4e9a16afef53 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:43.722108 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec4c5-3258-434a-8450-4e9a16afef53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:43.722349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eec4c5-3258-434a-8450-4e9a16afef53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:43.723093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eec4c5-3258-434a-8450-4e9a16afef53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:43.724735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eec4c5-3258-434a-8450-4e9a16afef53 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:43.736559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1121/2222] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:43.740450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0ccaa-3fb7-4d3c-b198-717be0f5e2d6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:43.758104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca0ccaa-3fb7-4d3c-b198-717be0f5e2d6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:43.765690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe45df-0be8-4104-86a3-b56bdc5d4754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:43.766456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe45df-0be8-4104-86a3-b56bdc5d4754 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:43.766709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe45df-0be8-4104-86a3-b56bdc5d4754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:43.766994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe45df-0be8-4104-86a3-b56bdc5d4754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:43.767730 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe45df-0be8-4104-86a3-b56bdc5d4754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:43.769454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe45df-0be8-4104-86a3-b56bdc5d4754 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:43.782062 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1102/2223] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:43.786204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eec4c5-3258-434a-8450-4e9a16afef53 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:43.796948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eec4c5-3258-434a-8450-4e9a16afef53 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:43.802384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b85a7-e2c1-424b-95dd-9a300c397cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:43.803362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b85a7-e2c1-424b-95dd-9a300c397cf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:43.803824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b85a7-e2c1-424b-95dd-9a300c397cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:43.803993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933b85a7-e2c1-424b-95dd-9a300c397cf2 admin admin] PATH_INFO: `/v3/projects/e74eb87483d94eb3b83505eb38698684/users/5b095272f202473c87e1a0b3401e67b4/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:43.821547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b85a7-e2c1-424b-95dd-9a300c397cf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e74eb87483d94eb3b83505eb38698684, user_id=5b095272f202473c87e1a0b3401e67b4, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:43.824425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933b85a7-e2c1-424b-95dd-9a300c397cf2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:43.841806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1122/2224] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:43 2022] PUT /identity/v3/projects/e74eb87483d94eb3b83505eb38698684/users/5b095272f202473c87e1a0b3401e67b4/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:43.853596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b8858-06f3-4b8a-b798-c66d087e8550 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:43.853891 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b8858-06f3-4b8a-b798-c66d087e8550 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:43.854253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b8858-06f3-4b8a-b798-c66d087e8550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:43.883036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-9a1b8858-06f3-4b8a-b798-c66d087e8550 None None] MFA Rules not processed for user `5b095272f202473c87e1a0b3401e67b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:43.884255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9a1b8858-06f3-4b8a-b798-c66d087e8550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:43.952426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1103/2225] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:43.956811 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b85a7-e2c1-424b-95dd-9a300c397cf2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:44.023753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933b85a7-e2c1-424b-95dd-9a300c397cf2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:44.028537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e09e4-75e3-47dc-a344-b55b49715e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:44.029722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e09e4-75e3-47dc-a344-b55b49715e1c admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:44.030022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e09e4-75e3-47dc-a344-b55b49715e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:44.030325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e09e4-75e3-47dc-a344-b55b49715e1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:44.031008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e09e4-75e3-47dc-a344-b55b49715e1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:44.032945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7e09e4-75e3-47dc-a344-b55b49715e1c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:44.053674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:44.066219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1123/2226] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:43 2022] POST /identity/v3/projects => generated 393 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:44.067219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b8858-06f3-4b8a-b798-c66d087e8550 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:44.079144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b8858-06f3-4b8a-b798-c66d087e8550 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:44.120240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:44.140408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf6b23-7e18-4002-b1a5-6a0beb22ada7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:44.141093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf6b23-7e18-4002-b1a5-6a0beb22ada7 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:44.141382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf6b23-7e18-4002-b1a5-6a0beb22ada7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:44.141663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf6b23-7e18-4002-b1a5-6a0beb22ada7 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:44.142396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf6b23-7e18-4002-b1a5-6a0beb22ada7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:44.144332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf6b23-7e18-4002-b1a5-6a0beb22ada7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:44.171582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1104/2227] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:44 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:44.172819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e09e4-75e3-47dc-a344-b55b49715e1c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:44.184378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7e09e4-75e3-47dc-a344-b55b49715e1c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:44.189203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a80051-3254-466b-875a-bd6ba5108280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:44.189893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a80051-3254-466b-875a-bd6ba5108280 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:44.190217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a80051-3254-466b-875a-bd6ba5108280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:44.190499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a80051-3254-466b-875a-bd6ba5108280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:44.191247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a80051-3254-466b-875a-bd6ba5108280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:44.192868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a80051-3254-466b-875a-bd6ba5108280 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:44.205963 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1124/2228] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:44.210290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf6b23-7e18-4002-b1a5-6a0beb22ada7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:44.222576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf6b23-7e18-4002-b1a5-6a0beb22ada7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:44.227909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5079cf-2647-47cd-9689-966d1070c387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:44.230884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5079cf-2647-47cd-9689-966d1070c387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:44.230884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5079cf-2647-47cd-9689-966d1070c387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:44.230884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5079cf-2647-47cd-9689-966d1070c387 admin admin] PATH_INFO: `/v3/projects/c7ae8d51dfc44f078b6237f85377b891/users/7f7d97cd041e4d10a73c54a040d25c61/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:44.252654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5079cf-2647-47cd-9689-966d1070c387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ae8d51dfc44f078b6237f85377b891, user_id=7f7d97cd041e4d10a73c54a040d25c61, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:44.257041 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5079cf-2647-47cd-9689-966d1070c387 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:44.287710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1105/2229] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:44 2022] PUT /identity/v3/projects/c7ae8d51dfc44f078b6237f85377b891/users/7f7d97cd041e4d10a73c54a040d25c61/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:44.289248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a80051-3254-466b-875a-bd6ba5108280 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:44.303833 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a80051-3254-466b-875a-bd6ba5108280 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:44.396079 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfd84b-c560-406e-86e5-819910e64d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:44.396870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd84b-c560-406e-86e5-819910e64d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:44.397229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd84b-c560-406e-86e5-819910e64d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:44.397579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd84b-c560-406e-86e5-819910e64d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:44.398494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfd84b-c560-406e-86e5-819910e64d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:44.400133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfd84b-c560-406e-86e5-819910e64d2f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:44.414077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1125/2230] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:44 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:44.417298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5079cf-2647-47cd-9689-966d1070c387 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:44.428504 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5079cf-2647-47cd-9689-966d1070c387 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:44.436293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6086e-5a65-45d0-8f34-75a3560fa541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:44.437208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6086e-5a65-45d0-8f34-75a3560fa541 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:44.438258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6086e-5a65-45d0-8f34-75a3560fa541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:44.441868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6086e-5a65-45d0-8f34-75a3560fa541 admin admin] PATH_INFO: `/v3/projects/c7ae8d51dfc44f078b6237f85377b891/users/7f7d97cd041e4d10a73c54a040d25c61/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:44.441868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6086e-5a65-45d0-8f34-75a3560fa541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7ae8d51dfc44f078b6237f85377b891, user_id=7f7d97cd041e4d10a73c54a040d25c61, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:44.443548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6086e-5a65-45d0-8f34-75a3560fa541 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:44.467980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1106/2231] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:44 2022] PUT /identity/v3/projects/c7ae8d51dfc44f078b6237f85377b891/users/7f7d97cd041e4d10a73c54a040d25c61/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:44.468714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1122b7-697f-4e69-8554-a180f7cd4be1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:44.469010 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1122b7-697f-4e69-8554-a180f7cd4be1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:44.469295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1122b7-697f-4e69-8554-a180f7cd4be1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:44.516163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-aa1122b7-697f-4e69-8554-a180f7cd4be1 None None] MFA Rules not processed for user `7f7d97cd041e4d10a73c54a040d25c61`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:44.518439 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aa1122b7-697f-4e69-8554-a180f7cd4be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:44.626225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1126/2232] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:44 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:44.638513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6086e-5a65-45d0-8f34-75a3560fa541 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:44.644006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-16a6086e-5a65-45d0-8f34-75a3560fa541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:44.733827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6086e-5a65-45d0-8f34-75a3560fa541 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:44.737685 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc36ce-a1c3-42c1-b714-f4d8be45dae9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:44.738537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc36ce-a1c3-42c1-b714-f4d8be45dae9 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:44.738823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc36ce-a1c3-42c1-b714-f4d8be45dae9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:44.739216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc36ce-a1c3-42c1-b714-f4d8be45dae9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:44.752348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc36ce-a1c3-42c1-b714-f4d8be45dae9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:44.752348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bc36ce-a1c3-42c1-b714-f4d8be45dae9 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:44.764048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1107/2233] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:44 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:44.816000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1122b7-697f-4e69-8554-a180f7cd4be1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:44.817474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aa1122b7-697f-4e69-8554-a180f7cd4be1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:44.837638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1122b7-697f-4e69-8554-a180f7cd4be1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:44.841516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e45f610-6ddc-454f-8f4b-9092e593182a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:44.842309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45f610-6ddc-454f-8f4b-9092e593182a service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:44.842507 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45f610-6ddc-454f-8f4b-9092e593182a service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:44.842782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45f610-6ddc-454f-8f4b-9092e593182a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:44.921885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45f610-6ddc-454f-8f4b-9092e593182a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:44.923474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e45f610-6ddc-454f-8f4b-9092e593182a service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:44.936893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1127/2234] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:45.687679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc36ce-a1c3-42c1-b714-f4d8be45dae9 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:45.689568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f8bc36ce-a1c3-42c1-b714-f4d8be45dae9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:45.775016 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc36ce-a1c3-42c1-b714-f4d8be45dae9 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:45.782771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:45.784438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:45.784438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:45.784618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] PATH_INFO: `/v3/users/ff5253df3ed345bd9adf722fdfee70ad` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:45.785977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff5253df3ed345bd9adf722fdfee70ad)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:45.789937 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:45.847431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:45.860121 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff5253df3ed345bd9adf722fdfee70ad'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:45.860892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff5253df3ed345bd9adf722fdfee70ad'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:45.861598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff5253df3ed345bd9adf722fdfee70ad'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:45.883324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff5253df3ed345bd9adf722fdfee70ad'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:45.891726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff5253df3ed345bd9adf722fdfee70ad'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:45.908730 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1108/2235] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:45 2022] DELETE /identity/v3/users/ff5253df3ed345bd9adf722fdfee70ad => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:45.913284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d999e-8254-46f9-afab-d9d21a22de3b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:45.913749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d999e-8254-46f9-afab-d9d21a22de3b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:45.913826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1d999e-8254-46f9-afab-d9d21a22de3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:45.939185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:45.947190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ab1d999e-8254-46f9-afab-d9d21a22de3b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:45.948476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ab1d999e-8254-46f9-afab-d9d21a22de3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:46.010625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1128/2236] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:46.020312 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:46.021795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:46.083784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194debd4-8cd7-4a48-bf4b-19c836bb8004 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:46.087502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53611532-49dc-4fc0-b9ca-ae1f92a26a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:46.088185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53611532-49dc-4fc0-b9ca-ae1f92a26a3e service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:46.088475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53611532-49dc-4fc0-b9ca-ae1f92a26a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:46.088753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53611532-49dc-4fc0-b9ca-ae1f92a26a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:46.097946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53611532-49dc-4fc0-b9ca-ae1f92a26a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:46.099468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53611532-49dc-4fc0-b9ca-ae1f92a26a3e service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:46.110346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1109/2237] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:46.416825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d999e-8254-46f9-afab-d9d21a22de3b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:46.418735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ab1d999e-8254-46f9-afab-d9d21a22de3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:46.431676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1d999e-8254-46f9-afab-d9d21a22de3b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:46.437057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:46.438599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:46.438888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:46.439208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] PATH_INFO: `/v3/projects/e77cdf2a35d840848340bce1a1908284` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:46.440467 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e77cdf2a35d840848340bce1a1908284)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:46.442839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:46.493474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e77cdf2a35d840848340bce1a1908284', 'request_id': 'req-44faa062-c3a9-4bfd-a39f-0683bd781312'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:46.501052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e77cdf2a35d840848340bce1a1908284', 'request_id': 'req-44faa062-c3a9-4bfd-a39f-0683bd781312'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:46.509923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e77cdf2a35d840848340bce1a1908284', 'request_id': 'req-44faa062-c3a9-4bfd-a39f-0683bd781312'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:46.523588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] The token cache is being invalidate because project e77cdf2a35d840848340bce1a1908284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:46.523879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:46.524577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-44faa062-c3a9-4bfd-a39f-0683bd781312 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:46.526148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1129/2238] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:46 2022] DELETE /identity/v3/projects/e77cdf2a35d840848340bce1a1908284 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:48.316163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53611532-49dc-4fc0-b9ca-ae1f92a26a3e service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:48.317589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-53611532-49dc-4fc0-b9ca-ae1f92a26a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:48.330228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53611532-49dc-4fc0-b9ca-ae1f92a26a3e service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:48.396099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:48.397252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:48.397252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:48.397544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] PATH_INFO: `/v3/users/5b095272f202473c87e1a0b3401e67b4` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:48.399072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b095272f202473c87e1a0b3401e67b4)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:48.401031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:48.449735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b095272f202473c87e1a0b3401e67b4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:48.450680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b095272f202473c87e1a0b3401e67b4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:48.451298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b095272f202473c87e1a0b3401e67b4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:48.461748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:48.464134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b095272f202473c87e1a0b3401e67b4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:48.472883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b095272f202473c87e1a0b3401e67b4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:48.487247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1110/2239] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:48 2022] DELETE /identity/v3/users/5b095272f202473c87e1a0b3401e67b4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:48.491683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea3e40-e193-4029-8974-6731a4ade38f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:48.492129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea3e40-e193-4029-8974-6731a4ade38f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:48.492246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cea3e40-e193-4029-8974-6731a4ade38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:48.510023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1cea3e40-e193-4029-8974-6731a4ade38f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:48.511258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1cea3e40-e193-4029-8974-6731a4ade38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:48.568552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1130/2240] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:48.578026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:48.579443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:48.634632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3aec3-7190-4e67-b526-0f41012b4c4b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:48.638127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c0166-c967-4f83-94ca-758523a706b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:48.638762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c0166-c967-4f83-94ca-758523a706b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:48.639044 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c0166-c967-4f83-94ca-758523a706b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:48.639319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c0166-c967-4f83-94ca-758523a706b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:48.647996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c0166-c967-4f83-94ca-758523a706b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:48.649398 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c0166-c967-4f83-94ca-758523a706b5 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:48.659244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1111/2241] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:48.927482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea3e40-e193-4029-8974-6731a4ade38f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:48.928907 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1cea3e40-e193-4029-8974-6731a4ade38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:48.940847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cea3e40-e193-4029-8974-6731a4ade38f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:48.945411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:48.946303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:48.946589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:48.946867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] PATH_INFO: `/v3/projects/e74eb87483d94eb3b83505eb38698684` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:48.948057 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e74eb87483d94eb3b83505eb38698684)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:48.949989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:48.983923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:49.003117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e74eb87483d94eb3b83505eb38698684', 'request_id': 'req-a3b111b8-6e94-4e70-898f-602049560946'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.008966 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e74eb87483d94eb3b83505eb38698684', 'request_id': 'req-a3b111b8-6e94-4e70-898f-602049560946'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.016779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e74eb87483d94eb3b83505eb38698684', 'request_id': 'req-a3b111b8-6e94-4e70-898f-602049560946'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.027954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] The token cache is being invalidate because project e74eb87483d94eb3b83505eb38698684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:49.028240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.029463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.031646 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1131/2242] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:48 2022] DELETE /identity/v3/projects/e74eb87483d94eb3b83505eb38698684 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:49.035600 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c0166-c967-4f83-94ca-758523a706b5 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:49.044994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c0166-c967-4f83-94ca-758523a706b5 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:49.100261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:49.101175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:49.101461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:49.101733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] PATH_INFO: `/v3/users/7f7d97cd041e4d10a73c54a040d25c61` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:49.102950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f7d97cd041e4d10a73c54a040d25c61)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:49.104863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:49.149355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f7d97cd041e4d10a73c54a040d25c61'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.150005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f7d97cd041e4d10a73c54a040d25c61'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.150635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f7d97cd041e4d10a73c54a040d25c61'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.162371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f7d97cd041e4d10a73c54a040d25c61'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.169348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-515f03a7-5edd-4d6a-ace0-d25cf573193e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f7d97cd041e4d10a73c54a040d25c61'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.177652 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:49.182753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1112/2243] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:49 2022] DELETE /identity/v3/users/7f7d97cd041e4d10a73c54a040d25c61 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:49.458401 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:49.461258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:49.475111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b111b8-6e94-4e70-898f-602049560946 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:49.550139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:49.552356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:49.552813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:49.553098 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] PATH_INFO: `/v3/projects/c7ae8d51dfc44f078b6237f85377b891` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:49.554429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7ae8d51dfc44f078b6237f85377b891)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:49.556414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:49.610840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7ae8d51dfc44f078b6237f85377b891', 'request_id': 'req-338a73b0-44d7-4d60-8c76-502d74ca6a30'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.616800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7ae8d51dfc44f078b6237f85377b891', 'request_id': 'req-338a73b0-44d7-4d60-8c76-502d74ca6a30'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.623804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7ae8d51dfc44f078b6237f85377b891', 'request_id': 'req-338a73b0-44d7-4d60-8c76-502d74ca6a30'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.636062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] The token cache is being invalidate because project c7ae8d51dfc44f078b6237f85377b891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:49.636356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.637002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-338a73b0-44d7-4d60-8c76-502d74ca6a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:49.638576 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1132/2244] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:49 2022] DELETE /identity/v3/projects/c7ae8d51dfc44f078b6237f85377b891 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:49.643250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ff566-3506-49ec-bd33-fa5151708639 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:49.643642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ff566-3506-49ec-bd33-fa5151708639 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:49.643772 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ff566-3506-49ec-bd33-fa5151708639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:49.661930 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-737ff566-3506-49ec-bd33-fa5151708639 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:49.663066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-737ff566-3506-49ec-bd33-fa5151708639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:49.726432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1113/2245] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:49.733913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71939e6-c46a-4614-9e25-a75da0bc2116 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:49.734251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71939e6-c46a-4614-9e25-a75da0bc2116 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:49.734531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71939e6-c46a-4614-9e25-a75da0bc2116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:49.752774 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b71939e6-c46a-4614-9e25-a75da0bc2116 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:49.754362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b71939e6-c46a-4614-9e25-a75da0bc2116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:49.762138 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1133/2246] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:49.766806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ff566-3506-49ec-bd33-fa5151708639 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:49.776764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ff566-3506-49ec-bd33-fa5151708639 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:49.782191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ee7ec-0097-4f7f-b169-63cb07ef28fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:49.782924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ee7ec-0097-4f7f-b169-63cb07ef28fb admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:49.783208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ee7ec-0097-4f7f-b169-63cb07ef28fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:49.783491 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ee7ec-0097-4f7f-b169-63cb07ef28fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:49.784291 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ee7ec-0097-4f7f-b169-63cb07ef28fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:49.784597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ee7ec-0097-4f7f-b169-63cb07ef28fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:49.786214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ee7ec-0097-4f7f-b169-63cb07ef28fb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:49.798388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1114/2247] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:06:49.802718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71939e6-c46a-4614-9e25-a75da0bc2116 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:49.811853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71939e6-c46a-4614-9e25-a75da0bc2116 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:49.816430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460e022-343e-4626-be62-67febf622594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:49.817071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460e022-343e-4626-be62-67febf622594 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:49.817351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460e022-343e-4626-be62-67febf622594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:49.817623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460e022-343e-4626-be62-67febf622594 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:49.818363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460e022-343e-4626-be62-67febf622594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:49.820238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460e022-343e-4626-be62-67febf622594 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:49.845373 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1134/2248] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:49 2022] POST /identity/v3/projects => generated 399 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:49.849383 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ee7ec-0097-4f7f-b169-63cb07ef28fb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:49.859364 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ee7ec-0097-4f7f-b169-63cb07ef28fb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:49.922415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a87684-032b-49b6-9cc9-2b03abbc7616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:49.923095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a87684-032b-49b6-9cc9-2b03abbc7616 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:49.923382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a87684-032b-49b6-9cc9-2b03abbc7616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:49.923659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a87684-032b-49b6-9cc9-2b03abbc7616 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:49.924584 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a87684-032b-49b6-9cc9-2b03abbc7616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:49.927544 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a87684-032b-49b6-9cc9-2b03abbc7616 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:49.948984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1115/2249] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:49 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:49.953592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460e022-343e-4626-be62-67febf622594 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:49.964028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460e022-343e-4626-be62-67febf622594 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:49.968565 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7449d0-6ab8-4d2d-bfae-29cdee9af13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:49.969322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7449d0-6ab8-4d2d-bfae-29cdee9af13f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:49.969592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7449d0-6ab8-4d2d-bfae-29cdee9af13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:49.969865 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7449d0-6ab8-4d2d-bfae-29cdee9af13f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:49.970619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7449d0-6ab8-4d2d-bfae-29cdee9af13f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:49.972215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7449d0-6ab8-4d2d-bfae-29cdee9af13f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:49.984075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1135/2250] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:49.988005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a87684-032b-49b6-9cc9-2b03abbc7616 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:49.997792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a87684-032b-49b6-9cc9-2b03abbc7616 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:50.004510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9d2aef-b832-4c39-91ec-a84ff96cc56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:50.005258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d2aef-b832-4c39-91ec-a84ff96cc56f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.005543 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d2aef-b832-4c39-91ec-a84ff96cc56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.005819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9d2aef-b832-4c39-91ec-a84ff96cc56f admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.006589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d2aef-b832-4c39-91ec-a84ff96cc56f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:50.008217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9d2aef-b832-4c39-91ec-a84ff96cc56f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:50.020277 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1116/2251] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:50.024433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7449d0-6ab8-4d2d-bfae-29cdee9af13f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:50.030847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:50.034928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7449d0-6ab8-4d2d-bfae-29cdee9af13f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:50.041151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974312b0-275b-44b5-b16b-81ab7da61e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:50.042307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974312b0-275b-44b5-b16b-81ab7da61e37 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.042671 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974312b0-275b-44b5-b16b-81ab7da61e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.043115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974312b0-275b-44b5-b16b-81ab7da61e37 admin admin] PATH_INFO: `/v3/projects/0298cb8b5e5f40edbee21db92dc2120e/users/e9d9eaa0378d4150b605c92b5e6c456a/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.058736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974312b0-275b-44b5-b16b-81ab7da61e37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0298cb8b5e5f40edbee21db92dc2120e, user_id=e9d9eaa0378d4150b605c92b5e6c456a, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:50.061451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974312b0-275b-44b5-b16b-81ab7da61e37 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:50.075663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1136/2252] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:50 2022] PUT /identity/v3/projects/0298cb8b5e5f40edbee21db92dc2120e/users/e9d9eaa0378d4150b605c92b5e6c456a/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:50.087995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d3f8a-1139-4818-8e57-092fd0a709a7 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.088417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d3f8a-1139-4818-8e57-092fd0a709a7 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.088869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d3f8a-1139-4818-8e57-092fd0a709a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.118966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b74d3f8a-1139-4818-8e57-092fd0a709a7 None None] MFA Rules not processed for user `e9d9eaa0378d4150b605c92b5e6c456a`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:50.120241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b74d3f8a-1139-4818-8e57-092fd0a709a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:50.177329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:50.189969 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1117/2253] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:50.192357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974312b0-275b-44b5-b16b-81ab7da61e37 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:50.255091 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974312b0-275b-44b5-b16b-81ab7da61e37 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:50.259748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09b176-94e6-43d0-b45c-ad6899be4fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:50.260403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09b176-94e6-43d0-b45c-ad6899be4fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.260688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09b176-94e6-43d0-b45c-ad6899be4fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.260990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09b176-94e6-43d0-b45c-ad6899be4fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.261670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09b176-94e6-43d0-b45c-ad6899be4fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:50.263591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a09b176-94e6-43d0-b45c-ad6899be4fd4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:50.297915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1137/2254] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:50 2022] POST /identity/v3/projects => generated 401 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:50.301929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d3f8a-1139-4818-8e57-092fd0a709a7 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:50.314095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d3f8a-1139-4818-8e57-092fd0a709a7 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:50.368553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8242f-66a3-4083-bc45-c60e613bd7b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:50.369216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8242f-66a3-4083-bc45-c60e613bd7b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.369497 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8242f-66a3-4083-bc45-c60e613bd7b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.369771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa8242f-66a3-4083-bc45-c60e613bd7b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.370511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8242f-66a3-4083-bc45-c60e613bd7b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:50.372406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa8242f-66a3-4083-bc45-c60e613bd7b6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:50.392804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1118/2255] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:50 2022] POST /identity/v3/users => generated 307 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:50.396722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09b176-94e6-43d0-b45c-ad6899be4fd4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:50.406938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09b176-94e6-43d0-b45c-ad6899be4fd4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:50.411510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b314240-e403-4350-bca1-537775463f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:50.412161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b314240-e403-4350-bca1-537775463f9f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.412445 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b314240-e403-4350-bca1-537775463f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.412722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b314240-e403-4350-bca1-537775463f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.413451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b314240-e403-4350-bca1-537775463f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:50.415071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b314240-e403-4350-bca1-537775463f9f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:50.426140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1138/2256] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:06:50.430007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8242f-66a3-4083-bc45-c60e613bd7b6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:50.440385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa8242f-66a3-4083-bc45-c60e613bd7b6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:50.444903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f47082-1b7b-4377-8467-87d044d8ec48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:50.445768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f47082-1b7b-4377-8467-87d044d8ec48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.446072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f47082-1b7b-4377-8467-87d044d8ec48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.446359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f47082-1b7b-4377-8467-87d044d8ec48 admin admin] PATH_INFO: `/v3/projects/2c94105fc66945658ace62b6e612203e/users/83d8f20369294e968072eb66324c9e06/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.460608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f47082-1b7b-4377-8467-87d044d8ec48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c94105fc66945658ace62b6e612203e, user_id=83d8f20369294e968072eb66324c9e06, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:50.463296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f47082-1b7b-4377-8467-87d044d8ec48 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:50.476091 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1119/2257] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:50 2022] PUT /identity/v3/projects/2c94105fc66945658ace62b6e612203e/users/83d8f20369294e968072eb66324c9e06/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:50.479989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b314240-e403-4350-bca1-537775463f9f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:50.490281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b314240-e403-4350-bca1-537775463f9f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:50.545204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6e0f5-05b8-40fb-a95e-1fba79ad3e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:50.545864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e0f5-05b8-40fb-a95e-1fba79ad3e7a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.546668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e0f5-05b8-40fb-a95e-1fba79ad3e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.546944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6e0f5-05b8-40fb-a95e-1fba79ad3e7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.547657 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6e0f5-05b8-40fb-a95e-1fba79ad3e7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:50.549234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6e0f5-05b8-40fb-a95e-1fba79ad3e7a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:50.559008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1139/2258] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:50.563702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f47082-1b7b-4377-8467-87d044d8ec48 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:50.573728 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f47082-1b7b-4377-8467-87d044d8ec48 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:50.578724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016a64b-6cf0-43ae-8a3c-086fdbb18904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:50.579583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016a64b-6cf0-43ae-8a3c-086fdbb18904 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.579863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016a64b-6cf0-43ae-8a3c-086fdbb18904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.580136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2016a64b-6cf0-43ae-8a3c-086fdbb18904 admin admin] PATH_INFO: `/v3/projects/2c94105fc66945658ace62b6e612203e/users/83d8f20369294e968072eb66324c9e06/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.582066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016a64b-6cf0-43ae-8a3c-086fdbb18904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c94105fc66945658ace62b6e612203e, user_id=83d8f20369294e968072eb66324c9e06, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:50.584669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2016a64b-6cf0-43ae-8a3c-086fdbb18904 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:50.595283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1120/2259] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:50 2022] PUT /identity/v3/projects/2c94105fc66945658ace62b6e612203e/users/83d8f20369294e968072eb66324c9e06/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:50.605063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d23718-991d-4e66-ac2b-e70fc9121879 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.605489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d23718-991d-4e66-ac2b-e70fc9121879 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.605876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d23718-991d-4e66-ac2b-e70fc9121879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.632897 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-13d23718-991d-4e66-ac2b-e70fc9121879 None None] MFA Rules not processed for user `83d8f20369294e968072eb66324c9e06`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:50.634136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-13d23718-991d-4e66-ac2b-e70fc9121879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:50.674236 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:50.705592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1140/2260] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:50 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:50.715980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016a64b-6cf0-43ae-8a3c-086fdbb18904 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:50.717808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2016a64b-6cf0-43ae-8a3c-086fdbb18904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:50.780075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2016a64b-6cf0-43ae-8a3c-086fdbb18904 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:50.783704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50158b0b-db04-4cb2-9d3f-da000cb2225e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:50.784340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50158b0b-db04-4cb2-9d3f-da000cb2225e service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.784621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50158b0b-db04-4cb2-9d3f-da000cb2225e service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.784892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50158b0b-db04-4cb2-9d3f-da000cb2225e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.793547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50158b0b-db04-4cb2-9d3f-da000cb2225e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:50.795007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50158b0b-db04-4cb2-9d3f-da000cb2225e service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:50.804231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1121/2261] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:50 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:50.875607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d23718-991d-4e66-ac2b-e70fc9121879 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:50.885496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d23718-991d-4e66-ac2b-e70fc9121879 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:50.889096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69625492-b03d-4dd5-927c-ad8a117d1ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:50.890380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69625492-b03d-4dd5-927c-ad8a117d1ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:50.890672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69625492-b03d-4dd5-927c-ad8a117d1ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:50.890955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69625492-b03d-4dd5-927c-ad8a117d1ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:50.947206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69625492-b03d-4dd5-927c-ad8a117d1ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:50.948675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69625492-b03d-4dd5-927c-ad8a117d1ad2 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:50.958005 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1141/2262] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:55.011833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50158b0b-db04-4cb2-9d3f-da000cb2225e service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:55.061522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:55.074061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50158b0b-db04-4cb2-9d3f-da000cb2225e service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:55.079063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:55.079922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:55.080204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:55.080480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] PATH_INFO: `/v3/users/e9d9eaa0378d4150b605c92b5e6c456a` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:55.081691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9d9eaa0378d4150b605c92b5e6c456a)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:55.083633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:55.129243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9d9eaa0378d4150b605c92b5e6c456a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.129891 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9d9eaa0378d4150b605c92b5e6c456a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.130523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9d9eaa0378d4150b605c92b5e6c456a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.142537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9d9eaa0378d4150b605c92b5e6c456a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.149330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9d9eaa0378d4150b605c92b5e6c456a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.161787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1122/2263] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:55 2022] DELETE /identity/v3/users/e9d9eaa0378d4150b605c92b5e6c456a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:55.166266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e16d5b-cbf4-45e5-8902-737f613146f9 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:55.166715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e16d5b-cbf4-45e5-8902-737f613146f9 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:55.166846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e16d5b-cbf4-45e5-8902-737f613146f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:55.183520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-47e16d5b-cbf4-45e5-8902-737f613146f9 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:55.184710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-47e16d5b-cbf4-45e5-8902-737f613146f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:55.241252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1142/2264] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:55.251213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:55.252635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:55.310522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cd86ea-5627-442c-942e-0986f4141c17 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:55.314639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d825e4-dc69-46f4-b78b-70350c68f705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:55.315282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d825e4-dc69-46f4-b78b-70350c68f705 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:55.315567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d825e4-dc69-46f4-b78b-70350c68f705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:55.315843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d825e4-dc69-46f4-b78b-70350c68f705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:55.324723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d825e4-dc69-46f4-b78b-70350c68f705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:55.326173 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d825e4-dc69-46f4-b78b-70350c68f705 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:55.336114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1123/2265] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:55.601645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e16d5b-cbf4-45e5-8902-737f613146f9 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:55.603214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-47e16d5b-cbf4-45e5-8902-737f613146f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:55.618753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e16d5b-cbf4-45e5-8902-737f613146f9 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:55.623430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:55.624920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:55.625203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:55.625474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] PATH_INFO: `/v3/projects/0298cb8b5e5f40edbee21db92dc2120e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:55.626794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0298cb8b5e5f40edbee21db92dc2120e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:55.629107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:55.677231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0298cb8b5e5f40edbee21db92dc2120e', 'request_id': 'req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.683031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0298cb8b5e5f40edbee21db92dc2120e', 'request_id': 'req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.690780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0298cb8b5e5f40edbee21db92dc2120e', 'request_id': 'req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.697865 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:55.703299 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] The token cache is being invalidate because project 0298cb8b5e5f40edbee21db92dc2120e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:55.703779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.704877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.706622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1143/2266] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:55 2022] DELETE /identity/v3/projects/0298cb8b5e5f40edbee21db92dc2120e => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:55.710821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d825e4-dc69-46f4-b78b-70350c68f705 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:55.720510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d825e4-dc69-46f4-b78b-70350c68f705 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:55.775615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:55.776480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:55.776761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:55.777035 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] PATH_INFO: `/v3/users/83d8f20369294e968072eb66324c9e06` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:55.778271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d8f20369294e968072eb66324c9e06)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:55.780192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:55.793275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:55.826963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d8f20369294e968072eb66324c9e06'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.827739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d8f20369294e968072eb66324c9e06'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.828686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d8f20369294e968072eb66324c9e06'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.847195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d8f20369294e968072eb66324c9e06'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.861656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5539dc9e-a008-4de6-bf2c-d40af74a1f34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d8f20369294e968072eb66324c9e06'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:55.875649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1124/2267] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:06:55 2022] DELETE /identity/v3/users/83d8f20369294e968072eb66324c9e06 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:56.145494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:56.146948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:56.160435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fa691-35e4-4e18-9ee9-38f6bfad63b1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:56.216161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:56.217020 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.217301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.217575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] PATH_INFO: `/v3/projects/2c94105fc66945658ace62b6e612203e` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.219354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c94105fc66945658ace62b6e612203e)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:56.221298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:56.265549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c94105fc66945658ace62b6e612203e', 'request_id': 'req-2986ed47-8e49-4307-ac8b-5e089273c949'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:56.271423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c94105fc66945658ace62b6e612203e', 'request_id': 'req-2986ed47-8e49-4307-ac8b-5e089273c949'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:56.278517 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c94105fc66945658ace62b6e612203e', 'request_id': 'req-2986ed47-8e49-4307-ac8b-5e089273c949'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:56.289817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] The token cache is being invalidate because project 2c94105fc66945658ace62b6e612203e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:06:56.290132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:56.290759 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2986ed47-8e49-4307-ac8b-5e089273c949 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:06:56.292275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1144/2268] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:06:56 2022] DELETE /identity/v3/projects/2c94105fc66945658ace62b6e612203e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:06:56.297285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3d23a-2b32-4a89-8d2a-426b07fa741f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.297777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3d23a-2b32-4a89-8d2a-426b07fa741f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.297879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b3d23a-2b32-4a89-8d2a-426b07fa741f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.315583 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-50b3d23a-2b32-4a89-8d2a-426b07fa741f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:56.316827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-50b3d23a-2b32-4a89-8d2a-426b07fa741f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:56.374769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1125/2269] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:56.382872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdc5dd-3d28-4988-afe8-2b89ec33a781 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.383166 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdc5dd-3d28-4988-afe8-2b89ec33a781 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.383447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffdc5dd-3d28-4988-afe8-2b89ec33a781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.403203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dffdc5dd-3d28-4988-afe8-2b89ec33a781 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:56.404415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dffdc5dd-3d28-4988-afe8-2b89ec33a781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:56.412967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1145/2270] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:56.417005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3d23a-2b32-4a89-8d2a-426b07fa741f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:56.426736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b3d23a-2b32-4a89-8d2a-426b07fa741f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:56.431336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23d88f-3b6a-4e45-b641-0032cfef922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:56.431981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23d88f-3b6a-4e45-b641-0032cfef922e admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.432261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23d88f-3b6a-4e45-b641-0032cfef922e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.432532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23d88f-3b6a-4e45-b641-0032cfef922e admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.433252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23d88f-3b6a-4e45-b641-0032cfef922e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:06:56.433559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23d88f-3b6a-4e45-b641-0032cfef922e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:56.435211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23d88f-3b6a-4e45-b641-0032cfef922e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:56.446714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1126/2271] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:06:56.450697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdc5dd-3d28-4988-afe8-2b89ec33a781 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:56.460233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffdc5dd-3d28-4988-afe8-2b89ec33a781 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:56.464848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f7618-7768-4e2c-9ac4-836cf1561a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:56.465494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f7618-7768-4e2c-9ac4-836cf1561a27 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.465779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f7618-7768-4e2c-9ac4-836cf1561a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.466079 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755f7618-7768-4e2c-9ac4-836cf1561a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.466775 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f7618-7768-4e2c-9ac4-836cf1561a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:56.469251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755f7618-7768-4e2c-9ac4-836cf1561a27 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:56.493632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1146/2272] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:56 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:56.497623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23d88f-3b6a-4e45-b641-0032cfef922e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:56.507713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23d88f-3b6a-4e45-b641-0032cfef922e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:56.565963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662e57d-2eb5-41de-ab83-32131a83bdf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:56.566669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662e57d-2eb5-41de-ab83-32131a83bdf0 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.566963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662e57d-2eb5-41de-ab83-32131a83bdf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.567242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d662e57d-2eb5-41de-ab83-32131a83bdf0 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.567947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662e57d-2eb5-41de-ab83-32131a83bdf0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:56.570006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d662e57d-2eb5-41de-ab83-32131a83bdf0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:56.594548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1127/2273] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:56 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:56.595848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f7618-7768-4e2c-9ac4-836cf1561a27 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:56.606580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755f7618-7768-4e2c-9ac4-836cf1561a27 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:56.611397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90284cd3-269b-4b55-a3f9-8ce2b54df530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:56.612085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90284cd3-269b-4b55-a3f9-8ce2b54df530 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.612428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90284cd3-269b-4b55-a3f9-8ce2b54df530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.612732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90284cd3-269b-4b55-a3f9-8ce2b54df530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.613399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90284cd3-269b-4b55-a3f9-8ce2b54df530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:56.615030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90284cd3-269b-4b55-a3f9-8ce2b54df530 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:56.627894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1147/2274] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:56.632209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662e57d-2eb5-41de-ab83-32131a83bdf0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:56.643233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d662e57d-2eb5-41de-ab83-32131a83bdf0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:56.647947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20287d-da6a-4d2e-b937-e4f8d11f18c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:56.648688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20287d-da6a-4d2e-b937-e4f8d11f18c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.649170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20287d-da6a-4d2e-b937-e4f8d11f18c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.649456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20287d-da6a-4d2e-b937-e4f8d11f18c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.650224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20287d-da6a-4d2e-b937-e4f8d11f18c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:56.651832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c20287d-da6a-4d2e-b937-e4f8d11f18c6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:56.654475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:56.676710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1128/2275] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:56.680050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90284cd3-269b-4b55-a3f9-8ce2b54df530 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:56.690667 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90284cd3-269b-4b55-a3f9-8ce2b54df530 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:56.695397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4d24b-717a-43ce-8b83-ae4d2fd3aba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:56.696272 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4d24b-717a-43ce-8b83-ae4d2fd3aba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.696551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4d24b-717a-43ce-8b83-ae4d2fd3aba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.696824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d4d24b-717a-43ce-8b83-ae4d2fd3aba2 admin admin] PATH_INFO: `/v3/projects/506a6bdab52f41a98831ebbf60e8bde7/users/b32d0a72650a4881901ec47aa8e0bbe4/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.712082 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4d24b-717a-43ce-8b83-ae4d2fd3aba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506a6bdab52f41a98831ebbf60e8bde7, user_id=b32d0a72650a4881901ec47aa8e0bbe4, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:56.714730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d4d24b-717a-43ce-8b83-ae4d2fd3aba2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:56.727093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1148/2276] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:56 2022] PUT /identity/v3/projects/506a6bdab52f41a98831ebbf60e8bde7/users/b32d0a72650a4881901ec47aa8e0bbe4/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:56.731962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ef1a7-dd06-4a90-af03-d39c3b928db5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.732251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ef1a7-dd06-4a90-af03-d39c3b928db5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.732527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ef1a7-dd06-4a90-af03-d39c3b928db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.749783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-242ef1a7-dd06-4a90-af03-d39c3b928db5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:56.750953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-242ef1a7-dd06-4a90-af03-d39c3b928db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:56.809259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1129/2277] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:56.819265 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4d24b-717a-43ce-8b83-ae4d2fd3aba2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:56.821015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-35d4d24b-717a-43ce-8b83-ae4d2fd3aba2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:56.848732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:56.883947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d4d24b-717a-43ce-8b83-ae4d2fd3aba2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:56.887542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3d8c-4fd6-42ee-96bd-edb0e160a2b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:56.888178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3d8c-4fd6-42ee-96bd-edb0e160a2b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:56.888459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3d8c-4fd6-42ee-96bd-edb0e160a2b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:56.888734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f3d8c-4fd6-42ee-96bd-edb0e160a2b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:56.898099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f3d8c-4fd6-42ee-96bd-edb0e160a2b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:56.899503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2f3d8c-4fd6-42ee-96bd-edb0e160a2b8 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:56.908687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1149/2278] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:57.505787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e14c8-0c9e-4289-91ec-bfe76a0b8e56 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:57.506850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e14c8-0c9e-4289-91ec-bfe76a0b8e56 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:57.507132 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636e14c8-0c9e-4289-91ec-bfe76a0b8e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:57.534394 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-636e14c8-0c9e-4289-91ec-bfe76a0b8e56 None None] MFA Rules not processed for user `b32d0a72650a4881901ec47aa8e0bbe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:57.535534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-636e14c8-0c9e-4289-91ec-bfe76a0b8e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:57.603309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1130/2279] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:06:57.607822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3d8c-4fd6-42ee-96bd-edb0e160a2b8 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:57.619390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f3d8c-4fd6-42ee-96bd-edb0e160a2b8 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:57.623967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c021b4-ec37-4306-b2df-cd15a3ff9f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:57.624614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c021b4-ec37-4306-b2df-cd15a3ff9f82 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:57.624894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c021b4-ec37-4306-b2df-cd15a3ff9f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:57.625169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c021b4-ec37-4306-b2df-cd15a3ff9f82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:57.625846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c021b4-ec37-4306-b2df-cd15a3ff9f82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:57.627754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c021b4-ec37-4306-b2df-cd15a3ff9f82 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:57.656074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1150/2280] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:06:57 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:06:57.660159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e14c8-0c9e-4289-91ec-bfe76a0b8e56 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:57.672245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636e14c8-0c9e-4289-91ec-bfe76a0b8e56 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:57.739675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb798b-a30e-4edb-bc54-4fe09d36824f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:57.740359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb798b-a30e-4edb-bc54-4fe09d36824f admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:57.740647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb798b-a30e-4edb-bc54-4fe09d36824f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:57.740926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fb798b-a30e-4edb-bc54-4fe09d36824f admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:57.741635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb798b-a30e-4edb-bc54-4fe09d36824f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:57.743620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fb798b-a30e-4edb-bc54-4fe09d36824f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:57.768886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1131/2281] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:06:57 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:06:57.770453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c021b4-ec37-4306-b2df-cd15a3ff9f82 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:57.781405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c021b4-ec37-4306-b2df-cd15a3ff9f82 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:57.786087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c8b1c-819a-48ad-929f-41efd74618bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:57.786690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c8b1c-819a-48ad-929f-41efd74618bb admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:57.786975 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c8b1c-819a-48ad-929f-41efd74618bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:57.787252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c8b1c-819a-48ad-929f-41efd74618bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:57.787999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c8b1c-819a-48ad-929f-41efd74618bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:57.789582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c8b1c-819a-48ad-929f-41efd74618bb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:57.801405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1151/2282] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:57.806011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb798b-a30e-4edb-bc54-4fe09d36824f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:57.816534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fb798b-a30e-4edb-bc54-4fe09d36824f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:57.821260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42264182-0baa-4968-9bdc-02b3f3c0b809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:57.822348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42264182-0baa-4968-9bdc-02b3f3c0b809 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:57.822484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42264182-0baa-4968-9bdc-02b3f3c0b809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:57.822745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42264182-0baa-4968-9bdc-02b3f3c0b809 admin admin] PATH_INFO: `/v3/projects/e9309afefbc5431791196708b2a8e129/users/45ff2a8fde534aba90ff46f126a397d7/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:57.838833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42264182-0baa-4968-9bdc-02b3f3c0b809 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9309afefbc5431791196708b2a8e129, user_id=45ff2a8fde534aba90ff46f126a397d7, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:57.841505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42264182-0baa-4968-9bdc-02b3f3c0b809 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:57.844352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:57.857998 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1132/2283] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:57 2022] PUT /identity/v3/projects/e9309afefbc5431791196708b2a8e129/users/45ff2a8fde534aba90ff46f126a397d7/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:57.862021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c8b1c-819a-48ad-929f-41efd74618bb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:57.872183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c8b1c-819a-48ad-929f-41efd74618bb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:57.927977 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9eefe-42cf-4012-bcf5-e3c67b1f0ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:57.928641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9eefe-42cf-4012-bcf5-e3c67b1f0ade admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:57.928928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9eefe-42cf-4012-bcf5-e3c67b1f0ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:57.929202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9eefe-42cf-4012-bcf5-e3c67b1f0ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:57.929917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9eefe-42cf-4012-bcf5-e3c67b1f0ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:57.931548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9eefe-42cf-4012-bcf5-e3c67b1f0ade admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:57.941910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1152/2284] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:06:57.946064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42264182-0baa-4968-9bdc-02b3f3c0b809 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:57.956955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42264182-0baa-4968-9bdc-02b3f3c0b809 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:57.961540 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedda456-e3e0-49b2-b630-140fe3671ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:57.962449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedda456-e3e0-49b2-b630-140fe3671ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:57.962737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedda456-e3e0-49b2-b630-140fe3671ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:57.963015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedda456-e3e0-49b2-b630-140fe3671ccb admin admin] PATH_INFO: `/v3/projects/e9309afefbc5431791196708b2a8e129/users/45ff2a8fde534aba90ff46f126a397d7/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:57.964967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedda456-e3e0-49b2-b630-140fe3671ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9309afefbc5431791196708b2a8e129, user_id=45ff2a8fde534aba90ff46f126a397d7, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:57.968287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedda456-e3e0-49b2-b630-140fe3671ccb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:57.978470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1133/2285] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:06:57 2022] PUT /identity/v3/projects/e9309afefbc5431791196708b2a8e129/users/45ff2a8fde534aba90ff46f126a397d7/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:06:58.577581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec35a3-0626-4f74-85c6-598c24728e0f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:58.577875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec35a3-0626-4f74-85c6-598c24728e0f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:58.578194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ec35a3-0626-4f74-85c6-598c24728e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:58.605218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a5ec35a3-0626-4f74-85c6-598c24728e0f None None] MFA Rules not processed for user `45ff2a8fde534aba90ff46f126a397d7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:58.606488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a5ec35a3-0626-4f74-85c6-598c24728e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:58.679712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1153/2286] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:06:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:58.691143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedda456-e3e0-49b2-b630-140fe3671ccb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:58.692981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fedda456-e3e0-49b2-b630-140fe3671ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:58.750465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedda456-e3e0-49b2-b630-140fe3671ccb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:58.754301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dffeb-4960-4c8a-bfa2-c5d820c53980 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:58.754949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dffeb-4960-4c8a-bfa2-c5d820c53980 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:58.755237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dffeb-4960-4c8a-bfa2-c5d820c53980 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:58.755511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6dffeb-4960-4c8a-bfa2-c5d820c53980 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:58.810027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6dffeb-4960-4c8a-bfa2-c5d820c53980 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:58.811592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6dffeb-4960-4c8a-bfa2-c5d820c53980 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:58.820358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1134/2287] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:06:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:06:59.743437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15558771-0260-4be3-a3b1-684909495804 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:59.743909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15558771-0260-4be3-a3b1-684909495804 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:59.744027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15558771-0260-4be3-a3b1-684909495804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:59.761541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-15558771-0260-4be3-a3b1-684909495804 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:06:59.762763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-15558771-0260-4be3-a3b1-684909495804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:59.771996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:06:59.818058 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1154/2288] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:06:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:06:59.859880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dffeb-4960-4c8a-bfa2-c5d820c53980 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:06:59.861283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ae6dffeb-4960-4c8a-bfa2-c5d820c53980 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:06:59.930370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6dffeb-4960-4c8a-bfa2-c5d820c53980 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:06:59.935337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81962b7-b12b-41ff-9fd0-e5c21b90956b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:06:59.935924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81962b7-b12b-41ff-9fd0-e5c21b90956b service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:06:59.936210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81962b7-b12b-41ff-9fd0-e5c21b90956b service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:06:59.936517 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81962b7-b12b-41ff-9fd0-e5c21b90956b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:06:59.944432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81962b7-b12b-41ff-9fd0-e5c21b90956b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:06:59.946005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81962b7-b12b-41ff-9fd0-e5c21b90956b service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:06:59.954697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1135/2289] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:06:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:07:01.241150 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15558771-0260-4be3-a3b1-684909495804 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:01.251179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15558771-0260-4be3-a3b1-684909495804 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:01.255519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4555da8e-5ee8-4dfa-9ca1-ddd5df4b8c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:01.256455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555da8e-5ee8-4dfa-9ca1-ddd5df4b8c6c service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:01.256942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555da8e-5ee8-4dfa-9ca1-ddd5df4b8c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:01.257428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4555da8e-5ee8-4dfa-9ca1-ddd5df4b8c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:01.273395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555da8e-5ee8-4dfa-9ca1-ddd5df4b8c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:01.275143 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4555da8e-5ee8-4dfa-9ca1-ddd5df4b8c6c service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:01.286011 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1155/2290] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:07:01 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:07:09.951498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=2,keystone:SELECT=87 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:07:11.281358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=24 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:07:15.648901 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7655e-61a5-47a3-88a6-81a48ae46695 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:15.649423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7655e-61a5-47a3-88a6-81a48ae46695 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:15.649516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7655e-61a5-47a3-88a6-81a48ae46695 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:15.651041 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1136/2291] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:07:15.661289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098c765-a821-4403-91a7-a1db8b43cea6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:15.661733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098c765-a821-4403-91a7-a1db8b43cea6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:15.661915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098c765-a821-4403-91a7-a1db8b43cea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:15.690251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f098c765-a821-4403-91a7-a1db8b43cea6 None None] MFA Rules not processed for user `19ce6f259c22441486bfc04811a271b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:07:15.691484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f098c765-a821-4403-91a7-a1db8b43cea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:15.805541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1156/2292] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:07:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:07:15.922932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7655e-61a5-47a3-88a6-81a48ae46695 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:15.924971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f3a7655e-61a5-47a3-88a6-81a48ae46695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:15.944756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7655e-61a5-47a3-88a6-81a48ae46695 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:15.956438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415d7b3-d164-4274-9ead-476b70fa4cc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:15.957231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415d7b3-d164-4274-9ead-476b70fa4cc9 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:15.957620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415d7b3-d164-4274-9ead-476b70fa4cc9 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:15.958002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2415d7b3-d164-4274-9ead-476b70fa4cc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:15.968100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415d7b3-d164-4274-9ead-476b70fa4cc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:15.971233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2415d7b3-d164-4274-9ead-476b70fa4cc9 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:15.976533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07831bc2-b0f3-46bd-aab0-215e29aacd55 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:15.976820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07831bc2-b0f3-46bd-aab0-215e29aacd55 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:15.977096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07831bc2-b0f3-46bd-aab0-215e29aacd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:15.995648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-07831bc2-b0f3-46bd-aab0-215e29aacd55 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:07:15.996848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-07831bc2-b0f3-46bd-aab0-215e29aacd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:16.001316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1137/2293] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:07:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:07:16.016493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1157/2294] 10.222.0.13 () {58 vars in 967 bytes} [Sun Jun 26 03:07:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:07:16.029492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415d7b3-d164-4274-9ead-476b70fa4cc9 service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:16.042100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2415d7b3-d164-4274-9ead-476b70fa4cc9 service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:16.049209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bafe81-f165-4201-b1f0-24802a5a2251 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:16.049862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafe81-f165-4201-b1f0-24802a5a2251 service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:16.050176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafe81-f165-4201-b1f0-24802a5a2251 service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:16.050453 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bafe81-f165-4201-b1f0-24802a5a2251 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:16.059553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bafe81-f165-4201-b1f0-24802a5a2251 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:16.061247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bafe81-f165-4201-b1f0-24802a5a2251 service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:16.070427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1138/2295] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:07:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:07:16.793250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07831bc2-b0f3-46bd-aab0-215e29aacd55 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:16.885913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07831bc2-b0f3-46bd-aab0-215e29aacd55 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:16.890928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:16.891829 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:16.892110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:16.892391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] PATH_INFO: `/v3/users/b32d0a72650a4881901ec47aa8e0bbe4` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:16.893580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b32d0a72650a4881901ec47aa8e0bbe4)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:16.895517 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:16.924354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:07:16.946203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b32d0a72650a4881901ec47aa8e0bbe4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:16.946861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b32d0a72650a4881901ec47aa8e0bbe4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:16.954381 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b32d0a72650a4881901ec47aa8e0bbe4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:16.962331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b32d0a72650a4881901ec47aa8e0bbe4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:16.962940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b32d0a72650a4881901ec47aa8e0bbe4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:16.981264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1158/2296] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:07:16 2022] DELETE /identity/v3/users/b32d0a72650a4881901ec47aa8e0bbe4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:07:17.155887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bafe81-f165-4201-b1f0-24802a5a2251 service swift] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:17.171393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bafe81-f165-4201-b1f0-24802a5a2251 service swift] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:17.243773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:17.244636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:17.244922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:17.245202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] PATH_INFO: `/v3/projects/506a6bdab52f41a98831ebbf60e8bde7` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:17.246556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=506a6bdab52f41a98831ebbf60e8bde7)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:17.248458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:17.294829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506a6bdab52f41a98831ebbf60e8bde7', 'request_id': 'req-0fb799fa-b442-47a1-9457-f371331895b1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.302672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506a6bdab52f41a98831ebbf60e8bde7', 'request_id': 'req-0fb799fa-b442-47a1-9457-f371331895b1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.308542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506a6bdab52f41a98831ebbf60e8bde7', 'request_id': 'req-0fb799fa-b442-47a1-9457-f371331895b1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.331721 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] The token cache is being invalidate because project 506a6bdab52f41a98831ebbf60e8bde7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:07:17.332011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.332631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.334176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1139/2297] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:07:17 2022] DELETE /identity/v3/projects/506a6bdab52f41a98831ebbf60e8bde7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:07:17.338084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:17.339626 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:17.352387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04de6c2-a6b4-4480-9fa8-aa35532f38a1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:17.407146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:17.407954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:17.408221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:17.408494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] PATH_INFO: `/v3/users/45ff2a8fde534aba90ff46f126a397d7` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:17.409680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45ff2a8fde534aba90ff46f126a397d7)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:17.411673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:17.457179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45ff2a8fde534aba90ff46f126a397d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.457822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45ff2a8fde534aba90ff46f126a397d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.464745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45ff2a8fde534aba90ff46f126a397d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.471731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45ff2a8fde534aba90ff46f126a397d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.472344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d87ae4f3-55b9-4f82-bdc0-39e9a9f6a5b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45ff2a8fde534aba90ff46f126a397d7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.489827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1159/2298] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:07:17 2022] DELETE /identity/v3/users/45ff2a8fde534aba90ff46f126a397d7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:07:17.667365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:17.668974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:17.713421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb799fa-b442-47a1-9457-f371331895b1 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:17.775019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:07:17.785985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:17.787526 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:17.788215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:17.788509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] PATH_INFO: `/v3/projects/e9309afefbc5431791196708b2a8e129` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:17.790229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9309afefbc5431791196708b2a8e129)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:17.792830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:17.850179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9309afefbc5431791196708b2a8e129', 'request_id': 'req-41b732d6-85fa-4578-8f62-535286121e8a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.859289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9309afefbc5431791196708b2a8e129', 'request_id': 'req-41b732d6-85fa-4578-8f62-535286121e8a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.865101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9309afefbc5431791196708b2a8e129', 'request_id': 'req-41b732d6-85fa-4578-8f62-535286121e8a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.877282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] The token cache is being invalidate because project e9309afefbc5431791196708b2a8e129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:07:17.877570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.878246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41b732d6-85fa-4578-8f62-535286121e8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:17.880041 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1140/2299] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:07:17 2022] DELETE /identity/v3/projects/e9309afefbc5431791196708b2a8e129 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:07:17.886304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e7580-fc0a-40e3-899b-0e32039cdc3a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:17.886824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e7580-fc0a-40e3-899b-0e32039cdc3a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:17.886918 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e7580-fc0a-40e3-899b-0e32039cdc3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:17.919142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:07:17.935029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9d6e7580-fc0a-40e3-899b-0e32039cdc3a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:07:17.936539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9d6e7580-fc0a-40e3-899b-0e32039cdc3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:17.996906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1160/2300] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:07:18.007020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e9c9-20d3-44cd-9290-acab970b356a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.007318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e9c9-20d3-44cd-9290-acab970b356a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.007596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad08e9c9-20d3-44cd-9290-acab970b356a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.026581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ad08e9c9-20d3-44cd-9290-acab970b356a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:07:18.027937 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ad08e9c9-20d3-44cd-9290-acab970b356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:18.037412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1141/2301] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:07:18.042375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e7580-fc0a-40e3-899b-0e32039cdc3a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:18.052741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e7580-fc0a-40e3-899b-0e32039cdc3a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:18.061528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702e00c-103d-4427-afe1-e7790ad0552e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:18.063102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702e00c-103d-4427-afe1-e7790ad0552e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.063588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702e00c-103d-4427-afe1-e7790ad0552e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.063962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7702e00c-103d-4427-afe1-e7790ad0552e admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.064806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702e00c-103d-4427-afe1-e7790ad0552e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:07:18.065203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702e00c-103d-4427-afe1-e7790ad0552e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:18.066933 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7702e00c-103d-4427-afe1-e7790ad0552e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:18.085140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1161/2302] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:07:18.089493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08e9c9-20d3-44cd-9290-acab970b356a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:18.101356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad08e9c9-20d3-44cd-9290-acab970b356a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:18.108495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0acebe-6485-4ada-987d-0b85d805d2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:18.109805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0acebe-6485-4ada-987d-0b85d805d2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.110323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0acebe-6485-4ada-987d-0b85d805d2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.110765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0acebe-6485-4ada-987d-0b85d805d2c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.111562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0acebe-6485-4ada-987d-0b85d805d2c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:18.113716 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0acebe-6485-4ada-987d-0b85d805d2c3 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:18.126320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702e00c-103d-4427-afe1-e7790ad0552e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:18.128063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7702e00c-103d-4427-afe1-e7790ad0552e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:18.143778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7702e00c-103d-4427-afe1-e7790ad0552e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:18.145968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1142/2303] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:07:18 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:07:18.149844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0acebe-6485-4ada-987d-0b85d805d2c3 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:18.204396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:18.205187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.205664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.205941 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] PATH_INFO: `/v3/users/c31dbb3dcd9a411ea125573c8189e00c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.207195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c31dbb3dcd9a411ea125573c8189e00c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:18.209105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:18.210247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0acebe-6485-4ada-987d-0b85d805d2c3 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:18.215080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa55df0-ede0-4e20-8967-04a21aaaea82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:18.215747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa55df0-ede0-4e20-8967-04a21aaaea82 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.216032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa55df0-ede0-4e20-8967-04a21aaaea82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.216316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa55df0-ede0-4e20-8967-04a21aaaea82 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.217027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa55df0-ede0-4e20-8967-04a21aaaea82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:18.218974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa55df0-ede0-4e20-8967-04a21aaaea82 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:18.240496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1143/2304] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:07:18 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:07:18.244576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa55df0-ede0-4e20-8967-04a21aaaea82 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:18.255310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa55df0-ede0-4e20-8967-04a21aaaea82 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:18.259974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd92f49-3e7c-4282-8531-8da4c4b0edc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:18.260317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c31dbb3dcd9a411ea125573c8189e00c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:18.260610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd92f49-3e7c-4282-8531-8da4c4b0edc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.260895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd92f49-3e7c-4282-8531-8da4c4b0edc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.260994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c31dbb3dcd9a411ea125573c8189e00c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:18.261169 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd92f49-3e7c-4282-8531-8da4c4b0edc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.261889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd92f49-3e7c-4282-8531-8da4c4b0edc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:18.263496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd92f49-3e7c-4282-8531-8da4c4b0edc5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:18.268407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c31dbb3dcd9a411ea125573c8189e00c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:18.273551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1144/2305] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:07:18.276145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c31dbb3dcd9a411ea125573c8189e00c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:18.276911 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c31dbb3dcd9a411ea125573c8189e00c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:18.277681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd92f49-3e7c-4282-8531-8da4c4b0edc5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:18.279512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8dd92f49-3e7c-4282-8531-8da4c4b0edc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:18.291612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd92f49-3e7c-4282-8531-8da4c4b0edc5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:18.294748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1162/2306] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:07:18 2022] DELETE /identity/v3/users/c31dbb3dcd9a411ea125573c8189e00c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:07:18.296722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d710d2b-0744-47f2-afac-7915199f7d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:18.297365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d710d2b-0744-47f2-afac-7915199f7d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.297644 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d710d2b-0744-47f2-afac-7915199f7d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.297918 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d710d2b-0744-47f2-afac-7915199f7d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.298728 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d710d2b-0744-47f2-afac-7915199f7d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:18.300240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d710d2b-0744-47f2-afac-7915199f7d20 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:18.313018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1145/2307] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:07:18.315778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:18.324354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:07:18.381905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267c8416-20cf-49ba-acc5-fff4726377d2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:18.386930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac61ce-4f97-43c0-ad23-ecae054064e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:18.387851 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac61ce-4f97-43c0-ad23-ecae054064e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.388119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac61ce-4f97-43c0-ad23-ecae054064e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.388392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac61ce-4f97-43c0-ad23-ecae054064e9 admin admin] PATH_INFO: `/v3/projects/f93abb7628324906baa8d45ce8d1a153/users/497fd3327d1441c8b3933bda95f673af/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.403531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac61ce-4f97-43c0-ad23-ecae054064e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f93abb7628324906baa8d45ce8d1a153, user_id=497fd3327d1441c8b3933bda95f673af, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:18.406986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac61ce-4f97-43c0-ad23-ecae054064e9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:18.422082 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1163/2308] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:07:18 2022] PUT /identity/v3/projects/f93abb7628324906baa8d45ce8d1a153/users/497fd3327d1441c8b3933bda95f673af/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:07:18.426880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769cb91-f211-400d-b0d8-2c8dc3c155d2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.427177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769cb91-f211-400d-b0d8-2c8dc3c155d2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.427458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a769cb91-f211-400d-b0d8-2c8dc3c155d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.444200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a769cb91-f211-400d-b0d8-2c8dc3c155d2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:07:18.445647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a769cb91-f211-400d-b0d8-2c8dc3c155d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:18.464384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:07:18.471247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac61ce-4f97-43c0-ad23-ecae054064e9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:18.473096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4cac61ce-4f97-43c0-ad23-ecae054064e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:18.520466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1146/2309] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:07:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:07:18.530211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769cb91-f211-400d-b0d8-2c8dc3c155d2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:18.532691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a769cb91-f211-400d-b0d8-2c8dc3c155d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:18.545414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac61ce-4f97-43c0-ad23-ecae054064e9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:18.551389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:18.552247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.552534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.552811 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] PATH_INFO: `/v3/projects/5db904f456b040d8a9f16b169118d21a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.553986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5db904f456b040d8a9f16b169118d21a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:18.556080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:18.602298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5db904f456b040d8a9f16b169118d21a', 'request_id': 'req-6ef8aefc-7bf0-47c4-b09c-e96484238f99'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:18.607994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5db904f456b040d8a9f16b169118d21a', 'request_id': 'req-6ef8aefc-7bf0-47c4-b09c-e96484238f99'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:18.614391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a769cb91-f211-400d-b0d8-2c8dc3c155d2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:18.616832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5db904f456b040d8a9f16b169118d21a', 'request_id': 'req-6ef8aefc-7bf0-47c4-b09c-e96484238f99'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:18.628130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] The token cache is being invalidate because project 5db904f456b040d8a9f16b169118d21a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:07:18.628679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:18.629326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-6ef8aefc-7bf0-47c4-b09c-e96484238f99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:07:18.630913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1164/2310] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:07:18 2022] DELETE /identity/v3/projects/5db904f456b040d8a9f16b169118d21a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:07:18.672787 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c4a62-be4e-465d-b4bf-1ce7f046656a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:18.673467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c4a62-be4e-465d-b4bf-1ce7f046656a service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:18.673754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c4a62-be4e-465d-b4bf-1ce7f046656a service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:18.674035 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c4a62-be4e-465d-b4bf-1ce7f046656a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:18.676211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-eb6c4a62-be4e-465d-b4bf-1ce7f046656a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:18.745089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c4a62-be4e-465d-b4bf-1ce7f046656a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:18.746491 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6c4a62-be4e-465d-b4bf-1ce7f046656a service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:18.758245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1147/2311] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:07:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:07:19.449347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a18966-fe12-4985-b861-5a25dfa4b84f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:19.449820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a18966-fe12-4985-b861-5a25dfa4b84f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:19.449915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a18966-fe12-4985-b861-5a25dfa4b84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:19.480097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-59a18966-fe12-4985-b861-5a25dfa4b84f None None] MFA Rules not processed for user `497fd3327d1441c8b3933bda95f673af`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:07:19.481329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-59a18966-fe12-4985-b861-5a25dfa4b84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:19.489635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:07:19.551734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1165/2312] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:07:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:07:19.562786 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c4a62-be4e-465d-b4bf-1ce7f046656a service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:07:19.564287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-eb6c4a62-be4e-465d-b4bf-1ce7f046656a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:19.574781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:07:19.622381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c4a62-be4e-465d-b4bf-1ce7f046656a service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:07:19.626129 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed31d14-cba0-4091-81e3-a68e43430aa8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:07:19.626800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31d14-cba0-4091-81e3-a68e43430aa8 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:19.627080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31d14-cba0-4091-81e3-a68e43430aa8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:19.627351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed31d14-cba0-4091-81e3-a68e43430aa8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:19.636004 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed31d14-cba0-4091-81e3-a68e43430aa8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:07:19.637438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed31d14-cba0-4091-81e3-a68e43430aa8 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:07:19.646678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1148/2313] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:07:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:07:20.995555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394740b2-b54d-4e61-97b5-59964d557297 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:20.996031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394740b2-b54d-4e61-97b5-59964d557297 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:20.996139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394740b2-b54d-4e61-97b5-59964d557297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:21.013990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-394740b2-b54d-4e61-97b5-59964d557297 None None] MFA Rules not processed for user `497fd3327d1441c8b3933bda95f673af`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:07:21.015298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-394740b2-b54d-4e61-97b5-59964d557297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:21.022109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1166/2314] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:07:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:07:22.832720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823b4c1-9d64-4a09-9ee9-bb136f255917 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:07:22.833244 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823b4c1-9d64-4a09-9ee9-bb136f255917 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:07:22.833355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823b4c1-9d64-4a09-9ee9-bb136f255917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:07:22.850620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6823b4c1-9d64-4a09-9ee9-bb136f255917 None None] MFA Rules not processed for user `a1313e0937b842059f1e01480c98351c`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:07:22.851927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6823b4c1-9d64-4a09-9ee9-bb136f255917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:07:22.908942 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1149/2315] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:07:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:07:31.008782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=26 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:07:32.901217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=47 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:08:02.618901 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582f3d8-8f04-44fc-ad05-f1af7bf743ff None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:02.619434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582f3d8-8f04-44fc-ad05-f1af7bf743ff None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:02.619434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c582f3d8-8f04-44fc-ad05-f1af7bf743ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:02.637264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c582f3d8-8f04-44fc-ad05-f1af7bf743ff None None] MFA Rules not processed for user `3b7a90ca5e374fc882b644e34e0a3015`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:08:02.638566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c582f3d8-8f04-44fc-ad05-f1af7bf743ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:02.696031 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1167/2316] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:08:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:08:03.823276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823b4c1-9d64-4a09-9ee9-bb136f255917 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:08:03.824714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6823b4c1-9d64-4a09-9ee9-bb136f255917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:03.916129 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823b4c1-9d64-4a09-9ee9-bb136f255917 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:08:03.921329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0280bd7-2d9e-4254-93a0-0d398a01303e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:08:03.922153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0280bd7-2d9e-4254-93a0-0d398a01303e service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:03.922438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0280bd7-2d9e-4254-93a0-0d398a01303e service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:03.922714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0280bd7-2d9e-4254-93a0-0d398a01303e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:03.924521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b0280bd7-2d9e-4254-93a0-0d398a01303e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:03.936402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0280bd7-2d9e-4254-93a0-0d398a01303e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:08:03.937972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0280bd7-2d9e-4254-93a0-0d398a01303e service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:08:03.948081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1150/2317] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:08:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:08:12.688772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=25 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:08:13.944208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=41 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:08:30.149876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d89df-b504-4407-8c28-052501ebc518 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:30.150368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d89df-b504-4407-8c28-052501ebc518 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:30.151054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d89df-b504-4407-8c28-052501ebc518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:30.217364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-125d89df-b504-4407-8c28-052501ebc518 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:08:30.218706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-125d89df-b504-4407-8c28-052501ebc518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:30.306282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1168/2318] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:08:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:08:30.367191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0280bd7-2d9e-4254-93a0-0d398a01303e service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:08:30.368596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b0280bd7-2d9e-4254-93a0-0d398a01303e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:30.461083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0280bd7-2d9e-4254-93a0-0d398a01303e service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:08:30.466116 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94294fa-9907-4f26-93c8-c38670c1eef4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:08:30.466770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94294fa-9907-4f26-93c8-c38670c1eef4 service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:30.467057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94294fa-9907-4f26-93c8-c38670c1eef4 service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:30.467336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94294fa-9907-4f26-93c8-c38670c1eef4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:30.476648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94294fa-9907-4f26-93c8-c38670c1eef4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:08:30.478326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94294fa-9907-4f26-93c8-c38670c1eef4 service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:08:30.486435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1151/2319] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:08:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 26 03:08:40.303249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=47 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:08:40.483072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=41 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:08:48.763772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70507fed-0a98-4027-95ed-2c6211b80a1b None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:48.764329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70507fed-0a98-4027-95ed-2c6211b80a1b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:48.764329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70507fed-0a98-4027-95ed-2c6211b80a1b None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:48.765748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1169/2320] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:08:48.771528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25256e-027a-4412-94b3-babdbdaa429e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:48.772002 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25256e-027a-4412-94b3-babdbdaa429e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:48.772100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25256e-027a-4412-94b3-babdbdaa429e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:48.832110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-be25256e-027a-4412-94b3-babdbdaa429e None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:08:48.833344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-be25256e-027a-4412-94b3-babdbdaa429e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:48.915635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1152/2321] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:08:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:08:48.927636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70507fed-0a98-4027-95ed-2c6211b80a1b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:08:48.929326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-70507fed-0a98-4027-95ed-2c6211b80a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:49.009354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70507fed-0a98-4027-95ed-2c6211b80a1b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:08:49.012950 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80b725a-fb87-4792-a1e5-8d6a415926e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0dc5b601421a41d1a30a10e86967a920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:08:49.013592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b725a-fb87-4792-a1e5-8d6a415926e4 service glance] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:49.013882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b725a-fb87-4792-a1e5-8d6a415926e4 service glance] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:49.014197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80b725a-fb87-4792-a1e5-8d6a415926e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:49.024256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b725a-fb87-4792-a1e5-8d6a415926e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:08:49.025614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80b725a-fb87-4792-a1e5-8d6a415926e4 service glance] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:08:49.036639 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1170/2322] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:08:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:08:49.101469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472d6d2-8ee8-4d6b-9d4d-c7ffc309b90c None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:49.101738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472d6d2-8ee8-4d6b-9d4d-c7ffc309b90c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:49.102021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7472d6d2-8ee8-4d6b-9d4d-c7ffc309b90c None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:49.103480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1153/2323] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:08:49.108422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33825c-bce8-40f2-8e42-dd85336f3867 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:49.108719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33825c-bce8-40f2-8e42-dd85336f3867 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:49.109004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33825c-bce8-40f2-8e42-dd85336f3867 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:49.126969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7d33825c-bce8-40f2-8e42-dd85336f3867 None None] MFA Rules not processed for user `8ff708d6758d4f7b913e763824f3b791`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:08:49.128204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7d33825c-bce8-40f2-8e42-dd85336f3867 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:49.134959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1171/2324] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:08:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:08:49.153459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472d6d2-8ee8-4d6b-9d4d-c7ffc309b90c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:08:49.155247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7472d6d2-8ee8-4d6b-9d4d-c7ffc309b90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:49.168734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7472d6d2-8ee8-4d6b-9d4d-c7ffc309b90c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:08:49.172201 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4ebf05-dff8-45e6-9c4c-8fb51d51e817 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:08:49.172827 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ebf05-dff8-45e6-9c4c-8fb51d51e817 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:49.173113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ebf05-dff8-45e6-9c4c-8fb51d51e817 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:49.173390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4ebf05-dff8-45e6-9c4c-8fb51d51e817 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:49.181073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ebf05-dff8-45e6-9c4c-8fb51d51e817 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:08:49.182524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4ebf05-dff8-45e6-9c4c-8fb51d51e817 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:08:49.190136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1154/2325] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:08:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:08:49.280544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be0546-abc8-4c32-95f9-2a11a3a415b1 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:49.280767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be0546-abc8-4c32-95f9-2a11a3a415b1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:49.281050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be0546-abc8-4c32-95f9-2a11a3a415b1 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:49.282492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1172/2326] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:08:49.288607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626353d-7389-4e9c-bcb1-51bfdf506dcb None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:49.288906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626353d-7389-4e9c-bcb1-51bfdf506dcb None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:49.289187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2626353d-7389-4e9c-bcb1-51bfdf506dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:49.308850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2626353d-7389-4e9c-bcb1-51bfdf506dcb None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:08:49.310463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2626353d-7389-4e9c-bcb1-51bfdf506dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:49.320887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1155/2327] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:08:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:08:49.332738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be0546-abc8-4c32-95f9-2a11a3a415b1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:08:49.334472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d3be0546-abc8-4c32-95f9-2a11a3a415b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:49.348369 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be0546-abc8-4c32-95f9-2a11a3a415b1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:08:49.352141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfde3a-3725-44b9-8d5c-83942df37edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:08:49.353032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfde3a-3725-44b9-8d5c-83942df37edf service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:49.353322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfde3a-3725-44b9-8d5c-83942df37edf service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:49.353597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfde3a-3725-44b9-8d5c-83942df37edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:49.367221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfde3a-3725-44b9-8d5c-83942df37edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:08:49.368862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfde3a-3725-44b9-8d5c-83942df37edf service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:08:49.382410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1173/2328] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:08:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:08:54.542977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866b668-177a-4e92-abea-1f864a1b151b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:54.543462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866b668-177a-4e92-abea-1f864a1b151b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:54.543582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4866b668-177a-4e92-abea-1f864a1b151b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:54.562497 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4866b668-177a-4e92-abea-1f864a1b151b None None] MFA Rules not processed for user `497fd3327d1441c8b3933bda95f673af`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:08:54.563729 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4866b668-177a-4e92-abea-1f864a1b151b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:54.571230 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1156/2329] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:08:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:08:55.941002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b425fd-a79e-478d-ab56-b15e550c9c81 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:55.941467 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b425fd-a79e-478d-ab56-b15e550c9c81 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:55.941577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b425fd-a79e-478d-ab56-b15e550c9c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:55.961814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c0b425fd-a79e-478d-ab56-b15e550c9c81 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:08:55.963060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c0b425fd-a79e-478d-ab56-b15e550c9c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:08:55.970414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1174/2330] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:08:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:08:55.982776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866b668-177a-4e92-abea-1f864a1b151b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:08:55.994734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4866b668-177a-4e92-abea-1f864a1b151b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:08:55.999645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe71a82-15e2-4923-ab98-a74e728ab9b8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:08:56.000287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71a82-15e2-4923-ab98-a74e728ab9b8 service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:08:56.000567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71a82-15e2-4923-ab98-a74e728ab9b8 service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:08:56.000842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe71a82-15e2-4923-ab98-a74e728ab9b8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:08:56.009675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe71a82-15e2-4923-ab98-a74e728ab9b8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:08:56.011266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe71a82-15e2-4923-ab98-a74e728ab9b8 service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:08:56.019790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1157/2331] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:08:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:09:05.956857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=47 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:09:06.015935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=68 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:09:31.616887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6449e18-b341-4b4e-a402-26f3265e3023 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:09:31.617425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6449e18-b341-4b4e-a402-26f3265e3023 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:09:31.617425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6449e18-b341-4b4e-a402-26f3265e3023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:09:31.634721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c6449e18-b341-4b4e-a402-26f3265e3023 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:09:31.635896 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c6449e18-b341-4b4e-a402-26f3265e3023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:09:31.642814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1175/2332] 10.222.0.13 () {58 vars in 967 bytes} [Sun Jun 26 03:09:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:09:31.664771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe71a82-15e2-4923-ab98-a74e728ab9b8 service swift] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:09:31.675202 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe71a82-15e2-4923-ab98-a74e728ab9b8 service swift] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:09:31.680759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69a131-250c-4f28-8d0b-546a6e8bd8f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:09:31.681389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69a131-250c-4f28-8d0b-546a6e8bd8f0 service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:09:31.681676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69a131-250c-4f28-8d0b-546a6e8bd8f0 service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:09:31.681956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a69a131-250c-4f28-8d0b-546a6e8bd8f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:09:31.690266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69a131-250c-4f28-8d0b-546a6e8bd8f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:09:31.691837 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a69a131-250c-4f28-8d0b-546a6e8bd8f0 service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:09:31.699928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1158/2333] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:09:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:09:41.630100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:09:41.696530 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:09:53.215635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6449e18-b341-4b4e-a402-26f3265e3023 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:09:53.234001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6449e18-b341-4b4e-a402-26f3265e3023 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:09:53.237625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaba23-66c8-45d1-a031-461f261eeb41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:09:53.238310 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaba23-66c8-45d1-a031-461f261eeb41 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:09:53.238592 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaba23-66c8-45d1-a031-461f261eeb41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:09:53.238867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faaba23-66c8-45d1-a031-461f261eeb41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:09:53.241144 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3faaba23-66c8-45d1-a031-461f261eeb41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:09:53.254320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaba23-66c8-45d1-a031-461f261eeb41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:09:53.256346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faaba23-66c8-45d1-a031-461f261eeb41 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:09:53.267818 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1176/2334] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:09:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:09:55.840898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69a131-250c-4f28-8d0b-546a6e8bd8f0 service swift] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:09:55.850273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a69a131-250c-4f28-8d0b-546a6e8bd8f0 service swift] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:09:55.853984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab70f96-5af9-4f6b-b448-ec7ccee653d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:09:55.854503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab70f96-5af9-4f6b-b448-ec7ccee653d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:09:55.854792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab70f96-5af9-4f6b-b448-ec7ccee653d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:09:55.855067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab70f96-5af9-4f6b-b448-ec7ccee653d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:09:55.856856 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fab70f96-5af9-4f6b-b448-ec7ccee653d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:09:55.870641 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab70f96-5af9-4f6b-b448-ec7ccee653d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:09:55.872054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab70f96-5af9-4f6b-b448-ec7ccee653d4 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:09:55.883705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1159/2335] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:09:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:10:03.262548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=10 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:10:04.726632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaba23-66c8-45d1-a031-461f261eeb41 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:04.728106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3faaba23-66c8-45d1-a031-461f261eeb41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:04.741436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faaba23-66c8-45d1-a031-461f261eeb41 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:04.746697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffe02f-fccd-43f6-a0ab-ac8ea9e352e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:04.747322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe02f-fccd-43f6-a0ab-ac8ea9e352e0 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:04.747601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe02f-fccd-43f6-a0ab-ac8ea9e352e0 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:04.747879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afffe02f-fccd-43f6-a0ab-ac8ea9e352e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:04.749531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-afffe02f-fccd-43f6-a0ab-ac8ea9e352e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:04.806570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffe02f-fccd-43f6-a0ab-ac8ea9e352e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:04.808435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afffe02f-fccd-43f6-a0ab-ac8ea9e352e0 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:04.817347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1177/2336] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:10:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:10:05.878207 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:10:14.813205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=28 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:10:15.882854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab70f96-5af9-4f6b-b448-ec7ccee653d4 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:15.893786 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab70f96-5af9-4f6b-b448-ec7ccee653d4 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:15.899315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c027f8-a737-4c58-94b0-9f3994d24a2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:15.899947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c027f8-a737-4c58-94b0-9f3994d24a2c service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:15.900232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c027f8-a737-4c58-94b0-9f3994d24a2c service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:15.900515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c027f8-a737-4c58-94b0-9f3994d24a2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:15.902300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a9c027f8-a737-4c58-94b0-9f3994d24a2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:15.912840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c027f8-a737-4c58-94b0-9f3994d24a2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:15.914434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c027f8-a737-4c58-94b0-9f3994d24a2c service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:15.923148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1160/2337] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:10:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:10:23.927240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffe02f-fccd-43f6-a0ab-ac8ea9e352e0 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:23.939792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afffe02f-fccd-43f6-a0ab-ac8ea9e352e0 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:23.944963 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a061fe-f8ef-496d-ac81-e197b85fe992 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:23.945623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a061fe-f8ef-496d-ac81-e197b85fe992 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:23.945910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a061fe-f8ef-496d-ac81-e197b85fe992 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:23.946243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a061fe-f8ef-496d-ac81-e197b85fe992 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:23.948503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-68a061fe-f8ef-496d-ac81-e197b85fe992 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:23.963300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a061fe-f8ef-496d-ac81-e197b85fe992 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:23.965061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a061fe-f8ef-496d-ac81-e197b85fe992 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:23.971387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c027f8-a737-4c58-94b0-9f3994d24a2c service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:23.980937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1178/2338] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:10:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:10:23.988947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c027f8-a737-4c58-94b0-9f3994d24a2c service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:23.994241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8828eb8-74df-4c61-91bd-9b720b761193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:23.994949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828eb8-74df-4c61-91bd-9b720b761193 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:23.995411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828eb8-74df-4c61-91bd-9b720b761193 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:23.995543 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8828eb8-74df-4c61-91bd-9b720b761193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:23.997424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e8828eb8-74df-4c61-91bd-9b720b761193 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:24.005927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a061fe-f8ef-496d-ac81-e197b85fe992 service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:24.007376 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-68a061fe-f8ef-496d-ac81-e197b85fe992 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:24.021142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a061fe-f8ef-496d-ac81-e197b85fe992 service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:24.026703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:24.029374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:24.029659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:24.029846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8828eb8-74df-4c61-91bd-9b720b761193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:24.029968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] PATH_INFO: `/v3/users/497fd3327d1441c8b3933bda95f673af` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:24.031267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=497fd3327d1441c8b3933bda95f673af)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:24.031824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8828eb8-74df-4c61-91bd-9b720b761193 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:24.033116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:24.042314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1161/2339] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:10:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:10:24.083417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '497fd3327d1441c8b3933bda95f673af'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:10:24.084111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '497fd3327d1441c8b3933bda95f673af'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:10:24.092394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '497fd3327d1441c8b3933bda95f673af'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:10:24.100710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '497fd3327d1441c8b3933bda95f673af'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:10:24.101352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c7947cc7-2b2e-41f2-808e-cd896d4d80d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '497fd3327d1441c8b3933bda95f673af'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:10:24.124289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1179/2340] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:10:24 2022] DELETE /identity/v3/users/497fd3327d1441c8b3933bda95f673af => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:10:24.299449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8828eb8-74df-4c61-91bd-9b720b761193 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:24.313829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8828eb8-74df-4c61-91bd-9b720b761193 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:24.374395 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:24.375260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:24.375543 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:24.375814 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] PATH_INFO: `/v3/projects/f93abb7628324906baa8d45ce8d1a153` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:24.377008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f93abb7628324906baa8d45ce8d1a153)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:24.379227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:24.426661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f93abb7628324906baa8d45ce8d1a153', 'request_id': 'req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:10:24.434728 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f93abb7628324906baa8d45ce8d1a153', 'request_id': 'req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:10:24.441026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f93abb7628324906baa8d45ce8d1a153', 'request_id': 'req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:10:24.453293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] The token cache is being invalidate because project f93abb7628324906baa8d45ce8d1a153 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:10:24.453805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:10:24.454524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:10:24.456103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1162/2341] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:10:24 2022] DELETE /identity/v3/projects/f93abb7628324906baa8d45ce8d1a153 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:10:34.116612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=35,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:10:34.449974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:10:39.565466 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ae6cb-bc59-4b39-b339-47b93f43db1b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:39.565986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ae6cb-bc59-4b39-b339-47b93f43db1b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:39.566116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ae6cb-bc59-4b39-b339-47b93f43db1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:39.582688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-144ae6cb-bc59-4b39-b339-47b93f43db1b None None] MFA Rules not processed for user `a1313e0937b842059f1e01480c98351c`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:10:39.583883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-144ae6cb-bc59-4b39-b339-47b93f43db1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:39.638644 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1180/2342] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:10:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:10:40.524373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:40.525828 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:40.597626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1c4d3-4a6f-4c0d-826c-c3fab2d56470 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:40.603689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c15b0-a0e9-44e7-beac-c1d591d43e51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:40.604339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c15b0-a0e9-44e7-beac-c1d591d43e51 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:40.604654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c15b0-a0e9-44e7-beac-c1d591d43e51 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:40.604933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9c15b0-a0e9-44e7-beac-c1d591d43e51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:40.606747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9e9c15b0-a0e9-44e7-beac-c1d591d43e51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:40.663082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c15b0-a0e9-44e7-beac-c1d591d43e51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:40.665143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9c15b0-a0e9-44e7-beac-c1d591d43e51 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:40.674032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1163/2343] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:10:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:10:45.955980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ae6cb-bc59-4b39-b339-47b93f43db1b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:45.958098 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-144ae6cb-bc59-4b39-b339-47b93f43db1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:45.977049 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ae6cb-bc59-4b39-b339-47b93f43db1b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:45.983705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff537c6-3794-41f5-b81c-7c4160aece28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:45.984637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff537c6-3794-41f5-b81c-7c4160aece28 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:45.984922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff537c6-3794-41f5-b81c-7c4160aece28 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:45.985197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff537c6-3794-41f5-b81c-7c4160aece28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:45.987035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9ff537c6-3794-41f5-b81c-7c4160aece28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:45.999681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff537c6-3794-41f5-b81c-7c4160aece28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:46.003319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff537c6-3794-41f5-b81c-7c4160aece28 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:46.016659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1181/2344] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:10:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:10:46.948784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c15b0-a0e9-44e7-beac-c1d591d43e51 service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:46.960219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9c15b0-a0e9-44e7-beac-c1d591d43e51 service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:46.966365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915b5efb-1bb5-4b3d-94c9-4c6d714e6b3e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:46.967019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b5efb-1bb5-4b3d-94c9-4c6d714e6b3e service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:46.967305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b5efb-1bb5-4b3d-94c9-4c6d714e6b3e service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:46.967580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915b5efb-1bb5-4b3d-94c9-4c6d714e6b3e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:46.969403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-915b5efb-1bb5-4b3d-94c9-4c6d714e6b3e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:46.983602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b5efb-1bb5-4b3d-94c9-4c6d714e6b3e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:46.985267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915b5efb-1bb5-4b3d-94c9-4c6d714e6b3e service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:46.997275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1164/2345] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:10:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:10:52.185852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff537c6-3794-41f5-b81c-7c4160aece28 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:52.198066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff537c6-3794-41f5-b81c-7c4160aece28 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:52.203722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca08258-5b5e-4f12-bd72-76345a801d73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:52.204390 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca08258-5b5e-4f12-bd72-76345a801d73 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:52.204676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca08258-5b5e-4f12-bd72-76345a801d73 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:52.204955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca08258-5b5e-4f12-bd72-76345a801d73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:52.206796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2ca08258-5b5e-4f12-bd72-76345a801d73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:52.270220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca08258-5b5e-4f12-bd72-76345a801d73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:52.271815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca08258-5b5e-4f12-bd72-76345a801d73 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:52.282111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1182/2346] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:10:52 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:10:55.876262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616e0c0-3938-476f-b4da-bd9c3d2fc196 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:55.876747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616e0c0-3938-476f-b4da-bd9c3d2fc196 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:55.876876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5616e0c0-3938-476f-b4da-bd9c3d2fc196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:55.894192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-5616e0c0-3938-476f-b4da-bd9c3d2fc196 None None] MFA Rules not processed for user `3b7a90ca5e374fc882b644e34e0a3015`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:10:55.895388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5616e0c0-3938-476f-b4da-bd9c3d2fc196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:55.902129 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1165/2347] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:10:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:10:56.206665 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca08258-5b5e-4f12-bd72-76345a801d73 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:56.208130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2ca08258-5b5e-4f12-bd72-76345a801d73 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:56.277033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca08258-5b5e-4f12-bd72-76345a801d73 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:56.280654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64d9f93-748e-4a5e-aba2-44aac4cd0525 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:56.281320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9f93-748e-4a5e-aba2-44aac4cd0525 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:56.281604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9f93-748e-4a5e-aba2-44aac4cd0525 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:56.281881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64d9f93-748e-4a5e-aba2-44aac4cd0525 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:56.283726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a64d9f93-748e-4a5e-aba2-44aac4cd0525 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:56.294721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d9f93-748e-4a5e-aba2-44aac4cd0525 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:56.296167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64d9f93-748e-4a5e-aba2-44aac4cd0525 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:56.304727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1183/2348] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:10:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:10:57.692234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616e0c0-3938-476f-b4da-bd9c3d2fc196 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:10:57.695417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5616e0c0-3938-476f-b4da-bd9c3d2fc196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:57.782094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5616e0c0-3938-476f-b4da-bd9c3d2fc196 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:10:57.787239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6101bf5-95ea-474f-ba47-4ddd737f393d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:10:57.787924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6101bf5-95ea-474f-ba47-4ddd737f393d service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:57.791353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6101bf5-95ea-474f-ba47-4ddd737f393d service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:57.791353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6101bf5-95ea-474f-ba47-4ddd737f393d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:57.792666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e6101bf5-95ea-474f-ba47-4ddd737f393d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:57.805370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6101bf5-95ea-474f-ba47-4ddd737f393d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:10:57.807498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6101bf5-95ea-474f-ba47-4ddd737f393d service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:10:57.830253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1166/2349] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:10:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:10:59.444794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa28e2-5b0c-462e-a095-cd9bc55257ee None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:10:59.445280 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa28e2-5b0c-462e-a095-cd9bc55257ee None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:10:59.445397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa28e2-5b0c-462e-a095-cd9bc55257ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:10:59.462699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-32aa28e2-5b0c-462e-a095-cd9bc55257ee None None] MFA Rules not processed for user `430d2efc691e4aae9f96409b081c7e96`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:10:59.463923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-32aa28e2-5b0c-462e-a095-cd9bc55257ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:10:59.470831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1184/2350] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:10:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:11:07.817832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:11:09.457246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=88 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:11:10.761853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6101bf5-95ea-474f-ba47-4ddd737f393d service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:11:10.777258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6101bf5-95ea-474f-ba47-4ddd737f393d service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:11:10.782757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b63f4-7a8b-4cb5-a91e-45072ff06532 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:11:10.783404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b63f4-7a8b-4cb5-a91e-45072ff06532 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:11:10.783717 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b63f4-7a8b-4cb5-a91e-45072ff06532 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:11:10.784003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27b63f4-7a8b-4cb5-a91e-45072ff06532 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:11:10.785778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a27b63f4-7a8b-4cb5-a91e-45072ff06532 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:11:10.798764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b63f4-7a8b-4cb5-a91e-45072ff06532 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:11:10.800331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27b63f4-7a8b-4cb5-a91e-45072ff06532 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:11:10.811550 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1167/2351] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:11:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:11:20.806954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:11:42.381449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6eed3-4601-4f14-a3ee-f61c4ebdfcf1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:11:42.381974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6eed3-4601-4f14-a3ee-f61c4ebdfcf1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:11:42.382104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6eed3-4601-4f14-a3ee-f61c4ebdfcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:11:42.399075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e9d6eed3-4601-4f14-a3ee-f61c4ebdfcf1 None None] MFA Rules not processed for user `95793ccb15844069890363d983f1680d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:11:42.400268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e9d6eed3-4601-4f14-a3ee-f61c4ebdfcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:11:42.453118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1185/2352] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:11:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:11:42.750287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b63f4-7a8b-4cb5-a91e-45072ff06532 service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:11:42.759830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27b63f4-7a8b-4cb5-a91e-45072ff06532 service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:11:42.763432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dbb5d-f81a-467c-8ff4-8b7f1a28023b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:11:42.764066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dbb5d-f81a-467c-8ff4-8b7f1a28023b service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:11:42.764352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dbb5d-f81a-467c-8ff4-8b7f1a28023b service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:11:42.764632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7dbb5d-f81a-467c-8ff4-8b7f1a28023b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:11:42.766471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4f7dbb5d-f81a-467c-8ff4-8b7f1a28023b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:11:42.776381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dbb5d-f81a-467c-8ff4-8b7f1a28023b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:11:42.777807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7dbb5d-f81a-467c-8ff4-8b7f1a28023b service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:11:42.785622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1168/2353] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:11:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:11:52.446417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=25 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:11:52.782153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:12:09.606889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6eed3-4601-4f14-a3ee-f61c4ebdfcf1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:09.608288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e9d6eed3-4601-4f14-a3ee-f61c4ebdfcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:12:09.620838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6eed3-4601-4f14-a3ee-f61c4ebdfcf1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:09.624354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959144c-726a-43dc-8964-b3b4e2af6c97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:09.625019 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959144c-726a-43dc-8964-b3b4e2af6c97 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:09.625301 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959144c-726a-43dc-8964-b3b4e2af6c97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:09.625575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959144c-726a-43dc-8964-b3b4e2af6c97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:09.627364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2959144c-726a-43dc-8964-b3b4e2af6c97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:12:09.688503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959144c-726a-43dc-8964-b3b4e2af6c97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:09.689910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959144c-726a-43dc-8964-b3b4e2af6c97 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:09.699875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1186/2354] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:12:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:12:12.907431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dbb5d-f81a-467c-8ff4-8b7f1a28023b service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:12.909007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4f7dbb5d-f81a-467c-8ff4-8b7f1a28023b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:12:12.923663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7dbb5d-f81a-467c-8ff4-8b7f1a28023b service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:12.928270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:12.929187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:12.929990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:12.930357 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] PATH_INFO: `/v3/users/a1313e0937b842059f1e01480c98351c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:12.931555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1313e0937b842059f1e01480c98351c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:12.933456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:12.978749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1313e0937b842059f1e01480c98351c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:12.979448 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1313e0937b842059f1e01480c98351c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:12.980059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1313e0937b842059f1e01480c98351c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:12.991612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1313e0937b842059f1e01480c98351c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:12.999524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0a7b63de-715b-4368-b2d4-b2342e0352d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1313e0937b842059f1e01480c98351c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:13.012669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1169/2355] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:12:12 2022] DELETE /identity/v3/users/a1313e0937b842059f1e01480c98351c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:12:13.186846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959144c-726a-43dc-8964-b3b4e2af6c97 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:13.198049 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959144c-726a-43dc-8964-b3b4e2af6c97 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:13.254525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:13.255396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.255677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.255954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] PATH_INFO: `/v3/projects/aa1dff9e9be549308ec0ad3a13810062` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.257131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa1dff9e9be549308ec0ad3a13810062)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:13.259424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:13.307756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa1dff9e9be549308ec0ad3a13810062', 'request_id': 'req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:13.313118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa1dff9e9be549308ec0ad3a13810062', 'request_id': 'req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:13.319997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa1dff9e9be549308ec0ad3a13810062', 'request_id': 'req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:13.330634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] The token cache is being invalidate because project aa1dff9e9be549308ec0ad3a13810062 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:12:13.330917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:13.331531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-aad3e2ac-bd30-4d1b-9e31-0608de699ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:13.333044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1187/2356] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:12:13 2022] DELETE /identity/v3/projects/aa1dff9e9be549308ec0ad3a13810062 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:12:13.345218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6624-23a5-4d81-9c34-076371fba2a0 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.345506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6624-23a5-4d81-9c34-076371fba2a0 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.345780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d6624-23a5-4d81-9c34-076371fba2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.363360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f17d6624-23a5-4d81-9c34-076371fba2a0 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:12:13.364568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f17d6624-23a5-4d81-9c34-076371fba2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:12:13.422425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1170/2357] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:12:13.430522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fae90-9a5c-488b-b2d7-7012015aeb0e None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.430814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fae90-9a5c-488b-b2d7-7012015aeb0e None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.431090 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0fae90-9a5c-488b-b2d7-7012015aeb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.449932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9a0fae90-9a5c-488b-b2d7-7012015aeb0e None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:12:13.451179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9a0fae90-9a5c-488b-b2d7-7012015aeb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:12:13.458949 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1188/2358] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:12:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:12:13.463398 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d6624-23a5-4d81-9c34-076371fba2a0 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:13.472985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d6624-23a5-4d81-9c34-076371fba2a0 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:13.478431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb5c04-e15a-4df5-b5be-793c9c345b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:13.479081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb5c04-e15a-4df5-b5be-793c9c345b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.479362 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb5c04-e15a-4df5-b5be-793c9c345b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.479639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb5c04-e15a-4df5-b5be-793c9c345b99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.480358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb5c04-e15a-4df5-b5be-793c9c345b99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:12:13.480724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb5c04-e15a-4df5-b5be-793c9c345b99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:13.482326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb5c04-e15a-4df5-b5be-793c9c345b99 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:13.493647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1171/2359] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:12:13.497669 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fae90-9a5c-488b-b2d7-7012015aeb0e None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:13.506989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0fae90-9a5c-488b-b2d7-7012015aeb0e None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:13.511542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c1064-a242-4ee6-85ba-6f42ffd54aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:13.512207 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c1064-a242-4ee6-85ba-6f42ffd54aaa admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.512488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c1064-a242-4ee6-85ba-6f42ffd54aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.512761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c1064-a242-4ee6-85ba-6f42ffd54aaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.513492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c1064-a242-4ee6-85ba-6f42ffd54aaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:13.515423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c1064-a242-4ee6-85ba-6f42ffd54aaa admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:13.529019 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:12:13.551407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1189/2360] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:12:13 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:12:13.555646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb5c04-e15a-4df5-b5be-793c9c345b99 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:13.565992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb5c04-e15a-4df5-b5be-793c9c345b99 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:13.620756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd29e1-eef8-4397-999e-10fabee6f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:13.621415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd29e1-eef8-4397-999e-10fabee6f9b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.621700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd29e1-eef8-4397-999e-10fabee6f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.621977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbd29e1-eef8-4397-999e-10fabee6f9b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.622698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd29e1-eef8-4397-999e-10fabee6f9b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:13.624612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbd29e1-eef8-4397-999e-10fabee6f9b8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:13.631358 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:12:13.646385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1172/2361] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:12:13 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:12:13.650952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c1064-a242-4ee6-85ba-6f42ffd54aaa admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:13.660973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c1064-a242-4ee6-85ba-6f42ffd54aaa admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:13.666179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d53a38-4d6b-4a0b-a337-1a96bf1215b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:13.666831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d53a38-4d6b-4a0b-a337-1a96bf1215b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.667112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d53a38-4d6b-4a0b-a337-1a96bf1215b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.667387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d53a38-4d6b-4a0b-a337-1a96bf1215b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.668103 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d53a38-4d6b-4a0b-a337-1a96bf1215b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:13.669684 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d53a38-4d6b-4a0b-a337-1a96bf1215b1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:13.680288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1190/2362] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:12:13.683927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd29e1-eef8-4397-999e-10fabee6f9b8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:13.693539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbd29e1-eef8-4397-999e-10fabee6f9b8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:13.699373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21734c17-a948-4aad-b354-4feb4c0533a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:13.700054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21734c17-a948-4aad-b354-4feb4c0533a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.700341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21734c17-a948-4aad-b354-4feb4c0533a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.700636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21734c17-a948-4aad-b354-4feb4c0533a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.701682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21734c17-a948-4aad-b354-4feb4c0533a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:13.703674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21734c17-a948-4aad-b354-4feb4c0533a4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:13.714027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1173/2363] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:12:13.718270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d53a38-4d6b-4a0b-a337-1a96bf1215b1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:13.727914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d53a38-4d6b-4a0b-a337-1a96bf1215b1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:13.732427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8326e2-d31e-44e5-b5cd-8a24e1c43b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:13.733293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8326e2-d31e-44e5-b5cd-8a24e1c43b25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.733574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8326e2-d31e-44e5-b5cd-8a24e1c43b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.733847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8326e2-d31e-44e5-b5cd-8a24e1c43b25 admin admin] PATH_INFO: `/v3/projects/987da9443a74424dbddc3976c1e19d43/users/755340db2c014b93a80571797b33ea92/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.749192 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8326e2-d31e-44e5-b5cd-8a24e1c43b25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=987da9443a74424dbddc3976c1e19d43, user_id=755340db2c014b93a80571797b33ea92, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:13.751915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8326e2-d31e-44e5-b5cd-8a24e1c43b25 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:13.766179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1191/2364] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:12:13 2022] PUT /identity/v3/projects/987da9443a74424dbddc3976c1e19d43/users/755340db2c014b93a80571797b33ea92/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:12:13.776992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcf17d-2356-46a9-b5e9-19daf903ddf9 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.777288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcf17d-2356-46a9-b5e9-19daf903ddf9 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.777597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcf17d-2356-46a9-b5e9-19daf903ddf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.803204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-82fcf17d-2356-46a9-b5e9-19daf903ddf9 None None] MFA Rules not processed for user `755340db2c014b93a80571797b33ea92`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:12:13.804884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-82fcf17d-2356-46a9-b5e9-19daf903ddf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:12:13.866549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1174/2365] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:12:13 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:12:13.878500 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8326e2-d31e-44e5-b5cd-8a24e1c43b25 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:13.880419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7f8326e2-d31e-44e5-b5cd-8a24e1c43b25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:12:13.946426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8326e2-d31e-44e5-b5cd-8a24e1c43b25 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:13.950115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f6750-1e76-45ce-b178-072e4623515f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:13.950913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f6750-1e76-45ce-b178-072e4623515f service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:13.951196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f6750-1e76-45ce-b178-072e4623515f service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:13.951475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f6750-1e76-45ce-b178-072e4623515f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:13.959971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f6750-1e76-45ce-b178-072e4623515f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:13.961459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f6750-1e76-45ce-b178-072e4623515f service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:13.970096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1192/2366] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:12:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:12:23.863553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=3,keystone:SELECT=39 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:12:23.966161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=51,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:12:59.219809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcf17d-2356-46a9-b5e9-19daf903ddf9 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:59.293975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcf17d-2356-46a9-b5e9-19daf903ddf9 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:59.305216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:59.306621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:59.306621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:59.307438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] PATH_INFO: `/v3/users/755340db2c014b93a80571797b33ea92` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:59.308506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=755340db2c014b93a80571797b33ea92)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:59.310463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:59.371233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '755340db2c014b93a80571797b33ea92'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:59.371936 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '755340db2c014b93a80571797b33ea92'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:59.372559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '755340db2c014b93a80571797b33ea92'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:59.398760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '755340db2c014b93a80571797b33ea92'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:59.407924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '755340db2c014b93a80571797b33ea92'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:59.422788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1175/2367] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:12:59 2022] DELETE /identity/v3/users/755340db2c014b93a80571797b33ea92 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:12:59.426960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eafbf7-4503-423c-9b24-4c6ab785cb1d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:59.427484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eafbf7-4503-423c-9b24-4c6ab785cb1d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:59.427593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eafbf7-4503-423c-9b24-4c6ab785cb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:59.450088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c9eafbf7-4503-423c-9b24-4c6ab785cb1d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:12:59.451913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c9eafbf7-4503-423c-9b24-4c6ab785cb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:12:59.530097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1193/2368] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:12:59.538742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:59.540284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:12:59.609395 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec2508-7ff3-4c7d-a91a-a7a19adcf737 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:59.613087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf038cd0-7328-422f-b962-012bcd203cb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:59.613736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038cd0-7328-422f-b962-012bcd203cb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:59.614022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038cd0-7328-422f-b962-012bcd203cb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:59.614332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf038cd0-7328-422f-b962-012bcd203cb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:59.623995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf038cd0-7328-422f-b962-012bcd203cb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:59.625433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf038cd0-7328-422f-b962-012bcd203cb6 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:59.636284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1176/2369] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:12:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:12:59.912011 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eafbf7-4503-423c-9b24-4c6ab785cb1d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:12:59.913701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c9eafbf7-4503-423c-9b24-4c6ab785cb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:12:59.927325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eafbf7-4503-423c-9b24-4c6ab785cb1d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:12:59.932206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:12:59.933640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:12:59.933640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:12:59.933930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] PATH_INFO: `/v3/projects/987da9443a74424dbddc3976c1e19d43` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:12:59.935266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] RBAC: Authorizing `identity:delete_project(project_id=987da9443a74424dbddc3976c1e19d43)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:12:59.937408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:12:59.991353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987da9443a74424dbddc3976c1e19d43', 'request_id': 'req-28f667af-7505-42fc-96bd-fbb805e983df'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:12:59.997012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987da9443a74424dbddc3976c1e19d43', 'request_id': 'req-28f667af-7505-42fc-96bd-fbb805e983df'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:00.005699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987da9443a74424dbddc3976c1e19d43', 'request_id': 'req-28f667af-7505-42fc-96bd-fbb805e983df'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:00.017335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] The token cache is being invalidate because project 987da9443a74424dbddc3976c1e19d43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:13:00.017626 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:00.018425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-28f667af-7505-42fc-96bd-fbb805e983df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:00.019971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1194/2370] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:12:59 2022] DELETE /identity/v3/projects/987da9443a74424dbddc3976c1e19d43 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:13:00.024807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09b115-bcba-493f-99ea-242826ad6efc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:00.025192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09b115-bcba-493f-99ea-242826ad6efc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:00.025535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09b115-bcba-493f-99ea-242826ad6efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:00.044043 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0d09b115-bcba-493f-99ea-242826ad6efc None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:00.045231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0d09b115-bcba-493f-99ea-242826ad6efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:00.062569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:00.112585 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1177/2371] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:13:00.120391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6938e-8fa2-492d-8069-b991e176e8a3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:00.120683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6938e-8fa2-492d-8069-b991e176e8a3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:00.120968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b6938e-8fa2-492d-8069-b991e176e8a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:00.139341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a5b6938e-8fa2-492d-8069-b991e176e8a3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:00.140561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a5b6938e-8fa2-492d-8069-b991e176e8a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:00.148871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1195/2372] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:00.153789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09b115-bcba-493f-99ea-242826ad6efc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:00.163518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09b115-bcba-493f-99ea-242826ad6efc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:00.168838 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505e4d8-b234-4314-aeb7-ae4a35384795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:00.169428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505e4d8-b234-4314-aeb7-ae4a35384795 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:00.169710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505e4d8-b234-4314-aeb7-ae4a35384795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:00.169982 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505e4d8-b234-4314-aeb7-ae4a35384795 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:00.171048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505e4d8-b234-4314-aeb7-ae4a35384795 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:13:00.171048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505e4d8-b234-4314-aeb7-ae4a35384795 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:00.172606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505e4d8-b234-4314-aeb7-ae4a35384795 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:00.188055 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1178/2373] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:13:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:13:00.189474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6938e-8fa2-492d-8069-b991e176e8a3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:00.199677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b6938e-8fa2-492d-8069-b991e176e8a3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:00.204841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f107e1-c5bc-419b-81f5-8ace177b5d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:00.205509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f107e1-c5bc-419b-81f5-8ace177b5d30 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:00.205791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f107e1-c5bc-419b-81f5-8ace177b5d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:00.206221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f107e1-c5bc-419b-81f5-8ace177b5d30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:00.206924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f107e1-c5bc-419b-81f5-8ace177b5d30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:00.208929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f107e1-c5bc-419b-81f5-8ace177b5d30 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:00.240123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1196/2374] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:13:00 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:13:00.244320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505e4d8-b234-4314-aeb7-ae4a35384795 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:00.254816 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505e4d8-b234-4314-aeb7-ae4a35384795 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:00.321014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf694e1b-6f3a-41cc-bf89-e20ebf1f8cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:00.321690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf694e1b-6f3a-41cc-bf89-e20ebf1f8cfc admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:00.321974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf694e1b-6f3a-41cc-bf89-e20ebf1f8cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:00.322303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf694e1b-6f3a-41cc-bf89-e20ebf1f8cfc admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:00.323539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf694e1b-6f3a-41cc-bf89-e20ebf1f8cfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:00.325479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf694e1b-6f3a-41cc-bf89-e20ebf1f8cfc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:00.346564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1179/2375] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:13:00 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:13:00.350372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f107e1-c5bc-419b-81f5-8ace177b5d30 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:00.360615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f107e1-c5bc-419b-81f5-8ace177b5d30 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:00.366355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc588ac-ebaa-45e7-8697-9223ab8359b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:00.367348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc588ac-ebaa-45e7-8697-9223ab8359b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:00.367810 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc588ac-ebaa-45e7-8697-9223ab8359b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:00.368310 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc588ac-ebaa-45e7-8697-9223ab8359b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:00.369314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc588ac-ebaa-45e7-8697-9223ab8359b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:00.371193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc588ac-ebaa-45e7-8697-9223ab8359b9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:00.382246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1197/2376] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:13:00.386906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf694e1b-6f3a-41cc-bf89-e20ebf1f8cfc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:00.400175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf694e1b-6f3a-41cc-bf89-e20ebf1f8cfc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:00.406344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8a5aa-0fb0-441a-a78f-fa40e4585421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:00.407098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a5aa-0fb0-441a-a78f-fa40e4585421 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:00.407663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a5aa-0fb0-441a-a78f-fa40e4585421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:00.408020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8a5aa-0fb0-441a-a78f-fa40e4585421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:00.408831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8a5aa-0fb0-441a-a78f-fa40e4585421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:00.411051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8a5aa-0fb0-441a-a78f-fa40e4585421 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:00.422169 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1180/2377] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:13:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:13:00.426644 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc588ac-ebaa-45e7-8697-9223ab8359b9 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:00.439482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc588ac-ebaa-45e7-8697-9223ab8359b9 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:00.446384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77218207-9e9c-46ef-9833-3c14814c867a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:00.447291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77218207-9e9c-46ef-9833-3c14814c867a admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:00.448407 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77218207-9e9c-46ef-9833-3c14814c867a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:00.448710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77218207-9e9c-46ef-9833-3c14814c867a admin admin] PATH_INFO: `/v3/projects/fbf9d500fd3a41cdb218a5f3377c68ea/users/052955a0039745a9b93cb81164501501/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:00.464867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77218207-9e9c-46ef-9833-3c14814c867a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf9d500fd3a41cdb218a5f3377c68ea, user_id=052955a0039745a9b93cb81164501501, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:00.467652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77218207-9e9c-46ef-9833-3c14814c867a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:00.487156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1198/2378] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:13:00 2022] PUT /identity/v3/projects/fbf9d500fd3a41cdb218a5f3377c68ea/users/052955a0039745a9b93cb81164501501/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:13:00.497916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a11a7-1719-40e5-83db-36987a6a5703 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:00.498307 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a11a7-1719-40e5-83db-36987a6a5703 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:00.498662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a11a7-1719-40e5-83db-36987a6a5703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:00.527661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d22a11a7-1719-40e5-83db-36987a6a5703 None None] MFA Rules not processed for user `052955a0039745a9b93cb81164501501`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:00.529080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d22a11a7-1719-40e5-83db-36987a6a5703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:00.603035 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1181/2379] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:00 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:00.613844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77218207-9e9c-46ef-9833-3c14814c867a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:00.615564 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-77218207-9e9c-46ef-9833-3c14814c867a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:00.642994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:00.685882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77218207-9e9c-46ef-9833-3c14814c867a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:00.689999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5761ff-c872-492b-ac33-e49df044ee9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:00.690706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5761ff-c872-492b-ac33-e49df044ee9a service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:00.690995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5761ff-c872-492b-ac33-e49df044ee9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:00.691276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5761ff-c872-492b-ac33-e49df044ee9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:00.700371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5761ff-c872-492b-ac33-e49df044ee9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:00.702063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5761ff-c872-492b-ac33-e49df044ee9a service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:00.712797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1199/2380] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:13:00 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:13:01.604904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbcbd0-e3b2-41be-b5c4-e241a8b21d5b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:01.605419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbcbd0-e3b2-41be-b5c4-e241a8b21d5b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:01.605516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfbcbd0-e3b2-41be-b5c4-e241a8b21d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:01.622648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-fdfbcbd0-e3b2-41be-b5c4-e241a8b21d5b None None] MFA Rules not processed for user `3b7a90ca5e374fc882b644e34e0a3015`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:01.623849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fdfbcbd0-e3b2-41be-b5c4-e241a8b21d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:01.635496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:01.680256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1182/2381] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:03.127747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3a4e8-d4e4-4706-aa7f-928d7fae7502 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:03.128248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3a4e8-d4e4-4706-aa7f-928d7fae7502 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:03.128346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df3a4e8-d4e4-4706-aa7f-928d7fae7502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:03.144887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-5df3a4e8-d4e4-4706-aa7f-928d7fae7502 None None] MFA Rules not processed for user `052955a0039745a9b93cb81164501501`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:03.146093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5df3a4e8-d4e4-4706-aa7f-928d7fae7502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:03.153311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1200/2382] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:04.161875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbcbd0-e3b2-41be-b5c4-e241a8b21d5b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:04.163425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fdfbcbd0-e3b2-41be-b5c4-e241a8b21d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:04.263350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfbcbd0-e3b2-41be-b5c4-e241a8b21d5b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:04.268782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905fb30-7c30-455d-a0f5-d62ed8e7b84e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:04.269250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905fb30-7c30-455d-a0f5-d62ed8e7b84e service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:04.269537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905fb30-7c30-455d-a0f5-d62ed8e7b84e service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:04.269819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b905fb30-7c30-455d-a0f5-d62ed8e7b84e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:04.271679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b905fb30-7c30-455d-a0f5-d62ed8e7b84e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:04.283685 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905fb30-7c30-455d-a0f5-d62ed8e7b84e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:04.285271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b905fb30-7c30-455d-a0f5-d62ed8e7b84e service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:04.296076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1183/2383] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:13:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:13:05.543804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2599b1-6765-45e7-a999-5f43906a1927 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:05.544373 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2599b1-6765-45e7-a999-5f43906a1927 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:05.545052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2599b1-6765-45e7-a999-5f43906a1927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:05.566276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6e2599b1-6765-45e7-a999-5f43906a1927 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:05.567514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6e2599b1-6765-45e7-a999-5f43906a1927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:05.633029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1201/2384] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:13:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:05.693419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905fb30-7c30-455d-a0f5-d62ed8e7b84e service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:05.695314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b905fb30-7c30-455d-a0f5-d62ed8e7b84e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:05.782704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b905fb30-7c30-455d-a0f5-d62ed8e7b84e service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:05.787676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8074768-01c3-4ce8-bb31-816dcab97670 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:05.789069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8074768-01c3-4ce8-bb31-816dcab97670 service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:05.789793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8074768-01c3-4ce8-bb31-816dcab97670 service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:05.790255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8074768-01c3-4ce8-bb31-816dcab97670 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:05.800289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8074768-01c3-4ce8-bb31-816dcab97670 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:05.803386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8074768-01c3-4ce8-bb31-816dcab97670 service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:05.817860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1184/2385] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:13:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:13:15.625675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=45 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:15.813675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=81 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:22.592683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307db87-8fdf-406f-bd36-e2a5628d48d4 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:22.593167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307db87-8fdf-406f-bd36-e2a5628d48d4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:22.593405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307db87-8fdf-406f-bd36-e2a5628d48d4 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:22.595004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1202/2386] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:13:22.603257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581acbaa-d546-4ac4-b13d-6090ec0a0256 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:22.603943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581acbaa-d546-4ac4-b13d-6090ec0a0256 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:22.603943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581acbaa-d546-4ac4-b13d-6090ec0a0256 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:22.628133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-581acbaa-d546-4ac4-b13d-6090ec0a0256 None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:22.630757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-581acbaa-d546-4ac4-b13d-6090ec0a0256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:22.726715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1185/2387] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:13:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:22.741088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307db87-8fdf-406f-bd36-e2a5628d48d4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:22.743034 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b307db87-8fdf-406f-bd36-e2a5628d48d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:22.818627 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307db87-8fdf-406f-bd36-e2a5628d48d4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:22.822494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92bada1-0b15-45b7-bf17-1536291259a6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0dc5b601421a41d1a30a10e86967a920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:22.823264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bada1-0b15-45b7-bf17-1536291259a6 service glance] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:22.823614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bada1-0b15-45b7-bf17-1536291259a6 service glance] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:22.823956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92bada1-0b15-45b7-bf17-1536291259a6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:22.835321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bada1-0b15-45b7-bf17-1536291259a6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:22.836876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92bada1-0b15-45b7-bf17-1536291259a6 service glance] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:22.848880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1203/2388] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:13:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:13:22.924152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160f7e2-4270-4dbd-b442-3101f3b23a11 None None] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:22.924588 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160f7e2-4270-4dbd-b442-3101f3b23a11 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:22.925008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3160f7e2-4270-4dbd-b442-3101f3b23a11 None None] PATH_INFO: `/` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:22.926639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1186/2389] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:13:22.931917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2361d2-ebf2-4286-a071-1239c5354e85 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:22.932929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2361d2-ebf2-4286-a071-1239c5354e85 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:22.933280 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2361d2-ebf2-4286-a071-1239c5354e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:22.953247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ea2361d2-ebf2-4286-a071-1239c5354e85 None None] MFA Rules not processed for user `8ff708d6758d4f7b913e763824f3b791`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:22.954628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ea2361d2-ebf2-4286-a071-1239c5354e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:23.007117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1204/2390] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:13:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:23.019649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160f7e2-4270-4dbd-b442-3101f3b23a11 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:23.021336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3160f7e2-4270-4dbd-b442-3101f3b23a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:23.035048 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3160f7e2-4270-4dbd-b442-3101f3b23a11 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:23.038160 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089474ad-528a-45b1-9dd3-ed54be82a6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:23.038796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089474ad-528a-45b1-9dd3-ed54be82a6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:23.039083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089474ad-528a-45b1-9dd3-ed54be82a6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:23.039362 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089474ad-528a-45b1-9dd3-ed54be82a6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:23.047128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089474ad-528a-45b1-9dd3-ed54be82a6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:23.048564 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089474ad-528a-45b1-9dd3-ed54be82a6e8 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:23.058674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1187/2391] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:13:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:13:23.152308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7970f4-4989-4d65-a913-17448badca64 None None] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:23.152672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7970f4-4989-4d65-a913-17448badca64 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:23.152954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7970f4-4989-4d65-a913-17448badca64 None None] PATH_INFO: `/` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:23.154716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1205/2392] 10.222.0.13 () {54 vars in 921 bytes} [Sun Jun 26 03:13:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 26 03:13:23.161246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9499-bca4-48b7-b227-5ab77bf3c097 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:23.161545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9499-bca4-48b7-b227-5ab77bf3c097 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:23.161825 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9499-bca4-48b7-b227-5ab77bf3c097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:23.181171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e25f9499-bca4-48b7-b227-5ab77bf3c097 None None] MFA Rules not processed for user `9ee0150dfb3f42709029e42d0fd9105d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:23.182524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e25f9499-bca4-48b7-b227-5ab77bf3c097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:23.192209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1188/2393] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Jun 26 03:13:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:13:23.202694 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7970f4-4989-4d65-a913-17448badca64 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:23.204370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-de7970f4-4989-4d65-a913-17448badca64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:23.218084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7970f4-4989-4d65-a913-17448badca64 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:23.221745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e30d0c-e4fb-4ac6-aebb-9cc03fba9562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:23.222410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e30d0c-e4fb-4ac6-aebb-9cc03fba9562 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:23.222691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e30d0c-e4fb-4ac6-aebb-9cc03fba9562 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:23.222965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e30d0c-e4fb-4ac6-aebb-9cc03fba9562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:23.233699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e30d0c-e4fb-4ac6-aebb-9cc03fba9562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:23.235135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e30d0c-e4fb-4ac6-aebb-9cc03fba9562 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:23.246060 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1206/2394] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:13:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:13:23.426324 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86620c69-42f8-41c5-8500-b3e230e4394d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:23.426621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86620c69-42f8-41c5-8500-b3e230e4394d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:23.426901 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86620c69-42f8-41c5-8500-b3e230e4394d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:23.444365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-86620c69-42f8-41c5-8500-b3e230e4394d None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:23.445589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-86620c69-42f8-41c5-8500-b3e230e4394d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:23.452907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1189/2395] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:13:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:23.465742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e30d0c-e4fb-4ac6-aebb-9cc03fba9562 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:23.480130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e30d0c-e4fb-4ac6-aebb-9cc03fba9562 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:23.485430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3494dd-9c64-4052-86c1-55e775c4f97e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:23.486112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3494dd-9c64-4052-86c1-55e775c4f97e service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:23.486503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3494dd-9c64-4052-86c1-55e775c4f97e service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:23.486783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3494dd-9c64-4052-86c1-55e775c4f97e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:23.495473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3494dd-9c64-4052-86c1-55e775c4f97e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:23.497072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3494dd-9c64-4052-86c1-55e775c4f97e service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:23.505533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1207/2396] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:13:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:13:24.023858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c64d8-6f6e-41c0-9ab3-e9d05b781a13 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:24.024285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c64d8-6f6e-41c0-9ab3-e9d05b781a13 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:24.024412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c64d8-6f6e-41c0-9ab3-e9d05b781a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:24.041967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c86c64d8-6f6e-41c0-9ab3-e9d05b781a13 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:24.043224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c86c64d8-6f6e-41c0-9ab3-e9d05b781a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:24.050655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1190/2397] 10.222.0.13 () {58 vars in 967 bytes} [Sun Jun 26 03:13:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:24.063629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3494dd-9c64-4052-86c1-55e775c4f97e service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:24.074729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3494dd-9c64-4052-86c1-55e775c4f97e service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:24.083747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eef38c-4ca3-44f7-ad38-97b23344f6c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:24.084406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eef38c-4ca3-44f7-ad38-97b23344f6c1 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:24.084686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eef38c-4ca3-44f7-ad38-97b23344f6c1 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:24.085754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32eef38c-4ca3-44f7-ad38-97b23344f6c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:24.094733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eef38c-4ca3-44f7-ad38-97b23344f6c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:24.096333 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32eef38c-4ca3-44f7-ad38-97b23344f6c1 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:24.104708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1208/2398] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:13:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:13:24.301768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143ac82-698e-4e03-96df-18e8adaf19be None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:24.302166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143ac82-698e-4e03-96df-18e8adaf19be None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:24.302454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0143ac82-698e-4e03-96df-18e8adaf19be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:24.320837 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0143ac82-698e-4e03-96df-18e8adaf19be None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:24.322071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0143ac82-698e-4e03-96df-18e8adaf19be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:24.329412 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1191/2399] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:13:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:24.341469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eef38c-4ca3-44f7-ad38-97b23344f6c1 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:24.351945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32eef38c-4ca3-44f7-ad38-97b23344f6c1 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:24.357611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b0394-91ef-4eca-999c-5e61fd1bb10e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:24.358382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b0394-91ef-4eca-999c-5e61fd1bb10e service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:24.358668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b0394-91ef-4eca-999c-5e61fd1bb10e service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:24.358947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b0394-91ef-4eca-999c-5e61fd1bb10e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:24.367282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b0394-91ef-4eca-999c-5e61fd1bb10e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:24.368913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b0394-91ef-4eca-999c-5e61fd1bb10e service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:24.377721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1209/2400] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:13:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:13:24.693403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415540e-364e-4835-ae4e-0d59a6d5ab30 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:24.693696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415540e-364e-4835-ae4e-0d59a6d5ab30 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:24.693977 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d415540e-364e-4835-ae4e-0d59a6d5ab30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:24.711481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d415540e-364e-4835-ae4e-0d59a6d5ab30 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:24.712694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d415540e-364e-4835-ae4e-0d59a6d5ab30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:24.720638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1192/2401] 10.222.0.13 () {58 vars in 967 bytes} [Sun Jun 26 03:13:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:24.733382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b0394-91ef-4eca-999c-5e61fd1bb10e service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:24.743600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b0394-91ef-4eca-999c-5e61fd1bb10e service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:24.748752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530e0f-7c56-43f5-943f-35901f069518 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:24.749392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530e0f-7c56-43f5-943f-35901f069518 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:24.749719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530e0f-7c56-43f5-943f-35901f069518 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:24.750005 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530e0f-7c56-43f5-943f-35901f069518 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:24.758543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530e0f-7c56-43f5-943f-35901f069518 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:24.760098 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530e0f-7c56-43f5-943f-35901f069518 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:24.768154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1210/2402] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:13:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:13:24.971910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc469a9b-3634-4a16-85e8-8db5825d06e2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:24.972282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc469a9b-3634-4a16-85e8-8db5825d06e2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:24.972685 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc469a9b-3634-4a16-85e8-8db5825d06e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:24.991889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dc469a9b-3634-4a16-85e8-8db5825d06e2 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:24.993090 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dc469a9b-3634-4a16-85e8-8db5825d06e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:25.001258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1193/2403] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:13:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:25.013460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530e0f-7c56-43f5-943f-35901f069518 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:25.024232 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530e0f-7c56-43f5-943f-35901f069518 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:25.029395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68a7c2f-eab7-4ab7-9562-2b9544e5cff5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:25.030279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a7c2f-eab7-4ab7-9562-2b9544e5cff5 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:25.030656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a7c2f-eab7-4ab7-9562-2b9544e5cff5 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:25.033426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a7c2f-eab7-4ab7-9562-2b9544e5cff5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:25.043798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a7c2f-eab7-4ab7-9562-2b9544e5cff5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:25.047705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68a7c2f-eab7-4ab7-9562-2b9544e5cff5 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:25.061870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1211/2404] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:13:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:13:27.005920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc469a9b-3634-4a16-85e8-8db5825d06e2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:27.071542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc469a9b-3634-4a16-85e8-8db5825d06e2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:27.076948 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:27.077907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:27.078307 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:27.078652 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] PATH_INFO: `/v3/users/052955a0039745a9b93cb81164501501` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:27.079922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] RBAC: Authorizing `identity:delete_user(user_id=052955a0039745a9b93cb81164501501)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:27.081940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:27.125778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:27.133722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052955a0039745a9b93cb81164501501'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:27.135670 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052955a0039745a9b93cb81164501501'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:27.136417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052955a0039745a9b93cb81164501501'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:27.153911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052955a0039745a9b93cb81164501501'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:27.163364 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052955a0039745a9b93cb81164501501'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:27.180181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1194/2405] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:13:27 2022] DELETE /identity/v3/users/052955a0039745a9b93cb81164501501 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:13:27.185429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160e59c-e36e-4409-ada9-c209c20e4b33 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:27.185914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160e59c-e36e-4409-ada9-c209c20e4b33 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:27.186013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160e59c-e36e-4409-ada9-c209c20e4b33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:27.205476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4160e59c-e36e-4409-ada9-c209c20e4b33 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:27.206722 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4160e59c-e36e-4409-ada9-c209c20e4b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:27.241591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:27.275194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1212/2406] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:13:27.284482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:27.285914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:27.347400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9232c663-3273-4901-b1cc-d0e92e176295 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:27.351218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377724f5-52c2-4455-9905-c9a89fb6bd32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:27.351980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377724f5-52c2-4455-9905-c9a89fb6bd32 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:27.352270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377724f5-52c2-4455-9905-c9a89fb6bd32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:27.352547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377724f5-52c2-4455-9905-c9a89fb6bd32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:27.362904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377724f5-52c2-4455-9905-c9a89fb6bd32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:27.364347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377724f5-52c2-4455-9905-c9a89fb6bd32 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:27.382348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1195/2407] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:13:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:13:27.553542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160e59c-e36e-4409-ada9-c209c20e4b33 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:27.555173 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4160e59c-e36e-4409-ada9-c209c20e4b33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:27.578762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160e59c-e36e-4409-ada9-c209c20e4b33 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:27.583723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:27.584597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:27.584880 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:27.585155 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] PATH_INFO: `/v3/projects/fbf9d500fd3a41cdb218a5f3377c68ea` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:27.586360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf9d500fd3a41cdb218a5f3377c68ea)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:27.590081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:27.646888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf9d500fd3a41cdb218a5f3377c68ea', 'request_id': 'req-89370f35-7d46-4291-b10f-38a1fc3c5d70'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:27.654910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf9d500fd3a41cdb218a5f3377c68ea', 'request_id': 'req-89370f35-7d46-4291-b10f-38a1fc3c5d70'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:27.664196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf9d500fd3a41cdb218a5f3377c68ea', 'request_id': 'req-89370f35-7d46-4291-b10f-38a1fc3c5d70'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:27.678004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] The token cache is being invalidate because project fbf9d500fd3a41cdb218a5f3377c68ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:13:27.678503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:27.679171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-89370f35-7d46-4291-b10f-38a1fc3c5d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:27.680904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1213/2408] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:13:27 2022] DELETE /identity/v3/projects/fbf9d500fd3a41cdb218a5f3377c68ea => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:13:27.686327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d826f-c1b0-4e60-b5a2-8ccca49f165b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:27.686619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d826f-c1b0-4e60-b5a2-8ccca49f165b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:27.686893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d826f-c1b0-4e60-b5a2-8ccca49f165b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:27.709688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e59d826f-c1b0-4e60-b5a2-8ccca49f165b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:27.711448 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e59d826f-c1b0-4e60-b5a2-8ccca49f165b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:27.797415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1196/2409] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:27.805484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddaa79-1b03-48ab-b8e1-62b7b413f5d1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:27.805834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddaa79-1b03-48ab-b8e1-62b7b413f5d1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:27.806194 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddaa79-1b03-48ab-b8e1-62b7b413f5d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:27.830347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e9ddaa79-1b03-48ab-b8e1-62b7b413f5d1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:27.831883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e9ddaa79-1b03-48ab-b8e1-62b7b413f5d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:27.843124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1214/2410] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:27.847258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d826f-c1b0-4e60-b5a2-8ccca49f165b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:27.859376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d826f-c1b0-4e60-b5a2-8ccca49f165b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:27.872273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058986a1-5595-43e3-9ac6-5b6589cc767a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:27.873088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058986a1-5595-43e3-9ac6-5b6589cc767a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:27.873802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058986a1-5595-43e3-9ac6-5b6589cc767a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:27.874028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058986a1-5595-43e3-9ac6-5b6589cc767a admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:27.874808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058986a1-5595-43e3-9ac6-5b6589cc767a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:13:27.875118 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058986a1-5595-43e3-9ac6-5b6589cc767a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:27.876755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058986a1-5595-43e3-9ac6-5b6589cc767a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:27.890931 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1197/2411] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:13:27.895484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddaa79-1b03-48ab-b8e1-62b7b413f5d1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:27.906403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddaa79-1b03-48ab-b8e1-62b7b413f5d1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:27.911165 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7dbd0-b19d-417e-bc3f-ff02da78351a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:27.917923 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7dbd0-b19d-417e-bc3f-ff02da78351a admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:27.922030 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7dbd0-b19d-417e-bc3f-ff02da78351a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:27.923023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7dbd0-b19d-417e-bc3f-ff02da78351a admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:27.923773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7dbd0-b19d-417e-bc3f-ff02da78351a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:27.925970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7dbd0-b19d-417e-bc3f-ff02da78351a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:27.958093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1215/2412] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:13:27 2022] POST /identity/v3/projects => generated 391 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:13:27.961437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058986a1-5595-43e3-9ac6-5b6589cc767a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:27.972521 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058986a1-5595-43e3-9ac6-5b6589cc767a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:28.029734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:28.048239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c39e4-43aa-4ddc-adc9-c2251fc3346a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:28.049051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c39e4-43aa-4ddc-adc9-c2251fc3346a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:28.049340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c39e4-43aa-4ddc-adc9-c2251fc3346a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:28.049619 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c39e4-43aa-4ddc-adc9-c2251fc3346a admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:28.050392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c39e4-43aa-4ddc-adc9-c2251fc3346a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:28.052518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382c39e4-43aa-4ddc-adc9-c2251fc3346a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:28.081783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1198/2413] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:13:27 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:13:28.085676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7dbd0-b19d-417e-bc3f-ff02da78351a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:28.096736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7dbd0-b19d-417e-bc3f-ff02da78351a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:28.101288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f87b2-c663-4a7c-b67c-5c9260ee1fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:28.101964 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f87b2-c663-4a7c-b67c-5c9260ee1fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:28.102327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f87b2-c663-4a7c-b67c-5c9260ee1fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:28.102590 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724f87b2-c663-4a7c-b67c-5c9260ee1fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:28.103316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f87b2-c663-4a7c-b67c-5c9260ee1fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:28.105089 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724f87b2-c663-4a7c-b67c-5c9260ee1fc4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:28.119210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1216/2414] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:13:28.120464 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c39e4-43aa-4ddc-adc9-c2251fc3346a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:28.132089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c39e4-43aa-4ddc-adc9-c2251fc3346a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:28.137793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd136d4-bb0f-4cb9-88b9-a2b95220dd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:28.138669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd136d4-bb0f-4cb9-88b9-a2b95220dd8d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:28.139049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd136d4-bb0f-4cb9-88b9-a2b95220dd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:28.139335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd136d4-bb0f-4cb9-88b9-a2b95220dd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:28.140070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd136d4-bb0f-4cb9-88b9-a2b95220dd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:28.141672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd136d4-bb0f-4cb9-88b9-a2b95220dd8d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:28.154080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1199/2415] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:13:28.158585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f87b2-c663-4a7c-b67c-5c9260ee1fc4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:28.169489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724f87b2-c663-4a7c-b67c-5c9260ee1fc4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:28.174493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcf6c4-913e-45de-af11-137cd8e18178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:28.175418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcf6c4-913e-45de-af11-137cd8e18178 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:28.175702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcf6c4-913e-45de-af11-137cd8e18178 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:28.175979 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fcf6c4-913e-45de-af11-137cd8e18178 admin admin] PATH_INFO: `/v3/projects/1e7ddec6b9764fe390fe606c42a2981b/users/da0064847004464e96c065320924f484/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:28.191628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcf6c4-913e-45de-af11-137cd8e18178 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e7ddec6b9764fe390fe606c42a2981b, user_id=da0064847004464e96c065320924f484, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:28.194305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fcf6c4-913e-45de-af11-137cd8e18178 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:28.211027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1217/2416] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:13:28 2022] PUT /identity/v3/projects/1e7ddec6b9764fe390fe606c42a2981b/users/da0064847004464e96c065320924f484/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:13:28.222403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ebc6-cf84-4e7a-b8c8-ad4ffd58af8c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:28.222694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ebc6-cf84-4e7a-b8c8-ad4ffd58af8c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:28.222974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae2ebc6-cf84-4e7a-b8c8-ad4ffd58af8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:28.251371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-aae2ebc6-cf84-4e7a-b8c8-ad4ffd58af8c None None] MFA Rules not processed for user `da0064847004464e96c065320924f484`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:28.252605 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-aae2ebc6-cf84-4e7a-b8c8-ad4ffd58af8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:28.330336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1200/2417] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:28.342812 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcf6c4-913e-45de-af11-137cd8e18178 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:28.344557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b9fcf6c4-913e-45de-af11-137cd8e18178 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:28.405451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fcf6c4-913e-45de-af11-137cd8e18178 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:28.409132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc70e7f-c799-4e49-b992-745ec9b6ffd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:28.409718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc70e7f-c799-4e49-b992-745ec9b6ffd7 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:28.410003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc70e7f-c799-4e49-b992-745ec9b6ffd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:28.410315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc70e7f-c799-4e49-b992-745ec9b6ffd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:28.416514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:28.419234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc70e7f-c799-4e49-b992-745ec9b6ffd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:28.421255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc70e7f-c799-4e49-b992-745ec9b6ffd7 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:28.433334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1218/2418] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:13:28 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:13:38.327197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:38.434942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:55.032108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2ebc6-cf84-4e7a-b8c8-ad4ffd58af8c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:55.054805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183016e6-dbb7-43fc-bc7c-8d8ce8d97dc6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:55.055309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183016e6-dbb7-43fc-bc7c-8d8ce8d97dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:55.055412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183016e6-dbb7-43fc-bc7c-8d8ce8d97dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:55.072828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-183016e6-dbb7-43fc-bc7c-8d8ce8d97dc6 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:55.074068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-183016e6-dbb7-43fc-bc7c-8d8ce8d97dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:55.094470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae2ebc6-cf84-4e7a-b8c8-ad4ffd58af8c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:55.098943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:55.099800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:55.100079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:55.100348 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] PATH_INFO: `/v3/users/da0064847004464e96c065320924f484` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:55.101529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=da0064847004464e96c065320924f484)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:55.103487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:55.130613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1219/2419] 10.222.0.13 () {58 vars in 967 bytes} [Sun Jun 26 03:13:55 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:13:55.144323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183016e6-dbb7-43fc-bc7c-8d8ce8d97dc6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:55.145716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-183016e6-dbb7-43fc-bc7c-8d8ce8d97dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:55.151903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da0064847004464e96c065320924f484'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:55.153081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da0064847004464e96c065320924f484'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:55.153724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da0064847004464e96c065320924f484'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:55.167833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da0064847004464e96c065320924f484'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:55.176971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dd398a2f-0043-4c87-a472-2dd0112cdbfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da0064847004464e96c065320924f484'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:55.189371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1201/2420] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:13:55 2022] DELETE /identity/v3/users/da0064847004464e96c065320924f484 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:13:55.193665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e37fb-bd72-461e-a42d-824eeeda232c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:55.193959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e37fb-bd72-461e-a42d-824eeeda232c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:55.194259 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e37fb-bd72-461e-a42d-824eeeda232c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:55.211203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2f2e37fb-bd72-461e-a42d-824eeeda232c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:55.213077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2f2e37fb-bd72-461e-a42d-824eeeda232c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:55.220241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183016e6-dbb7-43fc-bc7c-8d8ce8d97dc6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:55.270857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1202/2421] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:13:55.279325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cf59b-8cc0-406d-bba2-757bf64b76b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:55.279976 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf59b-8cc0-406d-bba2-757bf64b76b1 service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:55.280258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf59b-8cc0-406d-bba2-757bf64b76b1 service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:55.280545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291cf59b-8cc0-406d-bba2-757bf64b76b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:55.281017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e37fb-bd72-461e-a42d-824eeeda232c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:55.282397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2f2e37fb-bd72-461e-a42d-824eeeda232c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:55.282513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-291cf59b-8cc0-406d-bba2-757bf64b76b1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:55.303027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:55.331455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:55.340377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e37fb-bd72-461e-a42d-824eeeda232c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:55.343318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cf59b-8cc0-406d-bba2-757bf64b76b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:55.344003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a286a63-f9b8-4513-b5d0-b5354e8a722c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:55.344650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a286a63-f9b8-4513-b5d0-b5354e8a722c service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:55.344900 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291cf59b-8cc0-406d-bba2-757bf64b76b1 service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:55.345015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a286a63-f9b8-4513-b5d0-b5354e8a722c service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:55.345213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a286a63-f9b8-4513-b5d0-b5354e8a722c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:55.353293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1220/2422] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:13:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 26 03:13:55.354180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a286a63-f9b8-4513-b5d0-b5354e8a722c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:55.355739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a286a63-f9b8-4513-b5d0-b5354e8a722c service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:55.365714 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1203/2423] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:13:55.639983 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cf59b-8cc0-406d-bba2-757bf64b76b1 service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:55.641639 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-291cf59b-8cc0-406d-bba2-757bf64b76b1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:55.661376 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291cf59b-8cc0-406d-bba2-757bf64b76b1 service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:55.667585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:55.668795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:55.669345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:55.669697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] PATH_INFO: `/v3/projects/1e7ddec6b9764fe390fe606c42a2981b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:55.671003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e7ddec6b9764fe390fe606c42a2981b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:55.672958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:55.724041 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e7ddec6b9764fe390fe606c42a2981b', 'request_id': 'req-a9ea5c3d-9609-482a-858c-028b32c7a2bc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:55.730341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e7ddec6b9764fe390fe606c42a2981b', 'request_id': 'req-a9ea5c3d-9609-482a-858c-028b32c7a2bc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:55.737750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e7ddec6b9764fe390fe606c42a2981b', 'request_id': 'req-a9ea5c3d-9609-482a-858c-028b32c7a2bc'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:55.749533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] The token cache is being invalidate because project 1e7ddec6b9764fe390fe606c42a2981b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:13:55.749820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:55.750495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9ea5c3d-9609-482a-858c-028b32c7a2bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:13:55.752013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1221/2424] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:13:55 2022] DELETE /identity/v3/projects/1e7ddec6b9764fe390fe606c42a2981b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:13:55.757247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea39535-4d3f-4de1-9c4a-1ac0c562cf72 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:55.757540 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea39535-4d3f-4de1-9c4a-1ac0c562cf72 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:55.757812 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea39535-4d3f-4de1-9c4a-1ac0c562cf72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:55.775712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2ea39535-4d3f-4de1-9c4a-1ac0c562cf72 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:55.776903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2ea39535-4d3f-4de1-9c4a-1ac0c562cf72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:55.834433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1204/2425] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:13:55.842100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a39714-566e-4567-be05-392e5361a475 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:55.842395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a39714-566e-4567-be05-392e5361a475 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:55.842674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a39714-566e-4567-be05-392e5361a475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:55.872393 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-54a39714-566e-4567-be05-392e5361a475 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:55.873347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-54a39714-566e-4567-be05-392e5361a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:55.881409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1222/2426] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:55.885593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea39535-4d3f-4de1-9c4a-1ac0c562cf72 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:55.896648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea39535-4d3f-4de1-9c4a-1ac0c562cf72 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:55.902180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956ea14-e111-493b-92d8-846fd6f6bc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:55.902756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956ea14-e111-493b-92d8-846fd6f6bc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:55.903037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956ea14-e111-493b-92d8-846fd6f6bc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:55.903310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956ea14-e111-493b-92d8-846fd6f6bc8c admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:55.904028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956ea14-e111-493b-92d8-846fd6f6bc8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:13:55.904361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956ea14-e111-493b-92d8-846fd6f6bc8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:55.906687 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9956ea14-e111-493b-92d8-846fd6f6bc8c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:55.919635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1205/2427] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:13:55.923905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a39714-566e-4567-be05-392e5361a475 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:55.934354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a39714-566e-4567-be05-392e5361a475 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:55.940045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38e796-3536-466d-9dfb-c844480775d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:55.940247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38e796-3536-466d-9dfb-c844480775d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:55.941809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38e796-3536-466d-9dfb-c844480775d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:55.941809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d38e796-3536-466d-9dfb-c844480775d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:55.941809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38e796-3536-466d-9dfb-c844480775d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:55.943695 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d38e796-3536-466d-9dfb-c844480775d8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:55.970642 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1223/2428] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:13:55 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:13:55.974640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956ea14-e111-493b-92d8-846fd6f6bc8c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:55.985705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956ea14-e111-493b-92d8-846fd6f6bc8c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:56.047110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d33300-8fd5-4009-a92f-984abdf65b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:56.047783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d33300-8fd5-4009-a92f-984abdf65b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:56.048069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d33300-8fd5-4009-a92f-984abdf65b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:56.048354 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d33300-8fd5-4009-a92f-984abdf65b20 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:56.049041 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d33300-8fd5-4009-a92f-984abdf65b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:56.050985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d33300-8fd5-4009-a92f-984abdf65b20 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:56.074861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1206/2429] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:13:55 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:13:56.076576 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38e796-3536-466d-9dfb-c844480775d8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:56.087065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d38e796-3536-466d-9dfb-c844480775d8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:56.092555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65957-b83b-4e84-9d3d-80855ec08dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:56.093212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65957-b83b-4e84-9d3d-80855ec08dab admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:56.093490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65957-b83b-4e84-9d3d-80855ec08dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:56.093759 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65957-b83b-4e84-9d3d-80855ec08dab admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:56.094509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e65957-b83b-4e84-9d3d-80855ec08dab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:56.096104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e65957-b83b-4e84-9d3d-80855ec08dab admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:56.107379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1224/2430] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:13:56.111390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d33300-8fd5-4009-a92f-984abdf65b20 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:56.121341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d33300-8fd5-4009-a92f-984abdf65b20 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:56.125875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b79f80c-c19d-44a7-a027-8700cfb78c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:56.126558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79f80c-c19d-44a7-a027-8700cfb78c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:56.126847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79f80c-c19d-44a7-a027-8700cfb78c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:56.127124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b79f80c-c19d-44a7-a027-8700cfb78c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:56.127843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79f80c-c19d-44a7-a027-8700cfb78c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:56.129682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b79f80c-c19d-44a7-a027-8700cfb78c59 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:56.140276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1207/2431] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:13:56.144212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65957-b83b-4e84-9d3d-80855ec08dab admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:56.154596 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65957-b83b-4e84-9d3d-80855ec08dab admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:56.159133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d115d5-d764-42e4-95a3-88dd23f36deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:56.160032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d115d5-d764-42e4-95a3-88dd23f36deb admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:56.160316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d115d5-d764-42e4-95a3-88dd23f36deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:56.160594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d115d5-d764-42e4-95a3-88dd23f36deb admin admin] PATH_INFO: `/v3/projects/1fe0cd601688420d88ea051894cfdf2a/users/1042fb5384e14cd2bc7090f09db977c2/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:56.175442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d115d5-d764-42e4-95a3-88dd23f36deb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe0cd601688420d88ea051894cfdf2a, user_id=1042fb5384e14cd2bc7090f09db977c2, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:56.178110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d115d5-d764-42e4-95a3-88dd23f36deb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:56.190560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1225/2432] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:13:56 2022] PUT /identity/v3/projects/1fe0cd601688420d88ea051894cfdf2a/users/1042fb5384e14cd2bc7090f09db977c2/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:13:56.195519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305da22-a3dc-4df5-9ff9-7a182945002b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:56.195915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305da22-a3dc-4df5-9ff9-7a182945002b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:56.196218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c305da22-a3dc-4df5-9ff9-7a182945002b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:56.213160 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c305da22-a3dc-4df5-9ff9-7a182945002b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:13:56.214377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c305da22-a3dc-4df5-9ff9-7a182945002b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:56.239416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:13:56.274198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1208/2433] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:13:56.284239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d115d5-d764-42e4-95a3-88dd23f36deb admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:13:56.285918 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-28d115d5-d764-42e4-95a3-88dd23f36deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:13:56.346486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d115d5-d764-42e4-95a3-88dd23f36deb admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:13:56.350126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffd4a7-3260-41cd-bd35-cd061b5c9831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:13:56.350766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffd4a7-3260-41cd-bd35-cd061b5c9831 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:13:56.351051 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffd4a7-3260-41cd-bd35-cd061b5c9831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:13:56.351326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ffd4a7-3260-41cd-bd35-cd061b5c9831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:13:56.360904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffd4a7-3260-41cd-bd35-cd061b5c9831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:13:56.362359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ffd4a7-3260-41cd-bd35-cd061b5c9831 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:13:56.371814 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1226/2434] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:00.311771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd6da-c3a0-4c77-ba12-ab605552c3c5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:00.312316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd6da-c3a0-4c77-ba12-ab605552c3c5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:00.312446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2afd6da-c3a0-4c77-ba12-ab605552c3c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:00.340445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c2afd6da-c3a0-4c77-ba12-ab605552c3c5 None None] MFA Rules not processed for user `1042fb5384e14cd2bc7090f09db977c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:00.341676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c2afd6da-c3a0-4c77-ba12-ab605552c3c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:00.406679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1209/2435] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:00.411160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffd4a7-3260-41cd-bd35-cd061b5c9831 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:00.420763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ffd4a7-3260-41cd-bd35-cd061b5c9831 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:00.425248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751895b-07b7-48fb-8b60-957e5d5ea121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:00.425896 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751895b-07b7-48fb-8b60-957e5d5ea121 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:00.426212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751895b-07b7-48fb-8b60-957e5d5ea121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:00.426491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751895b-07b7-48fb-8b60-957e5d5ea121 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:00.427183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751895b-07b7-48fb-8b60-957e5d5ea121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:00.429098 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751895b-07b7-48fb-8b60-957e5d5ea121 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:00.437820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:00.460257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1227/2436] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:00 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:00.464816 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd6da-c3a0-4c77-ba12-ab605552c3c5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:00.475991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2afd6da-c3a0-4c77-ba12-ab605552c3c5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:00.537591 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcbe1b-7f83-4a9b-b1fa-a27beaad6d38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:00.538329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcbe1b-7f83-4a9b-b1fa-a27beaad6d38 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:00.538622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcbe1b-7f83-4a9b-b1fa-a27beaad6d38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:00.538932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcbe1b-7f83-4a9b-b1fa-a27beaad6d38 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:00.539650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcbe1b-7f83-4a9b-b1fa-a27beaad6d38 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:00.541603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcbe1b-7f83-4a9b-b1fa-a27beaad6d38 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:00.562916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1210/2437] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:00 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:00.566968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751895b-07b7-48fb-8b60-957e5d5ea121 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:00.578910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751895b-07b7-48fb-8b60-957e5d5ea121 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:00.583438 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fa9ff-751e-4f44-8fd5-7292981822f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:00.584097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fa9ff-751e-4f44-8fd5-7292981822f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:00.584429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fa9ff-751e-4f44-8fd5-7292981822f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:00.584658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3fa9ff-751e-4f44-8fd5-7292981822f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:00.585350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fa9ff-751e-4f44-8fd5-7292981822f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:00.586971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3fa9ff-751e-4f44-8fd5-7292981822f3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:00.598671 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1228/2438] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:14:00.603783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcbe1b-7f83-4a9b-b1fa-a27beaad6d38 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:00.614860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcbe1b-7f83-4a9b-b1fa-a27beaad6d38 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:00.632735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44df73e-9b3f-472e-a35f-668e529ecf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:00.633653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44df73e-9b3f-472e-a35f-668e529ecf9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:00.633936 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44df73e-9b3f-472e-a35f-668e529ecf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:00.634897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44df73e-9b3f-472e-a35f-668e529ecf9a admin admin] PATH_INFO: `/v3/projects/7068039fead9452e89458bf2ea06dfeb/users/f907c9b48f8141788f3fad618dc545ab/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:00.658590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44df73e-9b3f-472e-a35f-668e529ecf9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7068039fead9452e89458bf2ea06dfeb, user_id=f907c9b48f8141788f3fad618dc545ab, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:00.661422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44df73e-9b3f-472e-a35f-668e529ecf9a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:00.677380 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1211/2439] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:00 2022] PUT /identity/v3/projects/7068039fead9452e89458bf2ea06dfeb/users/f907c9b48f8141788f3fad618dc545ab/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:00.682008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fa9ff-751e-4f44-8fd5-7292981822f3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:00.699426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3fa9ff-751e-4f44-8fd5-7292981822f3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:00.766634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c87494b-d5dc-4301-b85f-b20334cc11c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:00.767446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87494b-d5dc-4301-b85f-b20334cc11c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:00.767878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87494b-d5dc-4301-b85f-b20334cc11c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:00.768368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c87494b-d5dc-4301-b85f-b20334cc11c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:00.769047 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87494b-d5dc-4301-b85f-b20334cc11c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:00.770816 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c87494b-d5dc-4301-b85f-b20334cc11c7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:00.782943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1229/2440] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:14:00.787066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44df73e-9b3f-472e-a35f-668e529ecf9a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:00.800997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44df73e-9b3f-472e-a35f-668e529ecf9a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:00.806403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281e1b8-4ad6-4290-8965-028db5b8003c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:00.807393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281e1b8-4ad6-4290-8965-028db5b8003c admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:00.807884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281e1b8-4ad6-4290-8965-028db5b8003c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:00.808076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281e1b8-4ad6-4290-8965-028db5b8003c admin admin] PATH_INFO: `/v3/projects/7068039fead9452e89458bf2ea06dfeb/users/f907c9b48f8141788f3fad618dc545ab/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:00.810241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281e1b8-4ad6-4290-8965-028db5b8003c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7068039fead9452e89458bf2ea06dfeb, user_id=f907c9b48f8141788f3fad618dc545ab, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:00.813017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281e1b8-4ad6-4290-8965-028db5b8003c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:00.827339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1212/2441] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:00 2022] PUT /identity/v3/projects/7068039fead9452e89458bf2ea06dfeb/users/f907c9b48f8141788f3fad618dc545ab/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:05.129756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1e98b-84df-4eb4-a5ac-f7244197b01c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:05.130623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1e98b-84df-4eb4-a5ac-f7244197b01c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:05.130623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa1e98b-84df-4eb4-a5ac-f7244197b01c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:05.158717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-aaa1e98b-84df-4eb4-a5ac-f7244197b01c None None] MFA Rules not processed for user `f907c9b48f8141788f3fad618dc545ab`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:05.159893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aaa1e98b-84df-4eb4-a5ac-f7244197b01c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:05.229503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1230/2442] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:05.241267 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281e1b8-4ad6-4290-8965-028db5b8003c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:05.243094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3281e1b8-4ad6-4290-8965-028db5b8003c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:05.257321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:05.305013 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281e1b8-4ad6-4290-8965-028db5b8003c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:05.309154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6ed99-11e4-4cb1-b39e-278404501b3b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0dc5b601421a41d1a30a10e86967a920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:05.309806 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6ed99-11e4-4cb1-b39e-278404501b3b service glance] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:05.310122 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6ed99-11e4-4cb1-b39e-278404501b3b service glance] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:05.310403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6ed99-11e4-4cb1-b39e-278404501b3b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:05.367026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6ed99-11e4-4cb1-b39e-278404501b3b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:05.368985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6ed99-11e4-4cb1-b39e-278404501b3b service glance] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:05.378450 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1213/2443] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:05.485643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8354add-c067-4ce1-b311-1a63dea9c1a7 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:05.486271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8354add-c067-4ce1-b311-1a63dea9c1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:05.487083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8354add-c067-4ce1-b311-1a63dea9c1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:05.521174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f8354add-c067-4ce1-b311-1a63dea9c1a7 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:05.522464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f8354add-c067-4ce1-b311-1a63dea9c1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:05.592488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1231/2444] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:14:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:05.614835 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6ed99-11e4-4cb1-b39e-278404501b3b service glance] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:05.616329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-03a6ed99-11e4-4cb1-b39e-278404501b3b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:05.695100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6ed99-11e4-4cb1-b39e-278404501b3b service glance] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:05.700294 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0e103-18b9-4104-83d1-9ad4560fa1b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:05.700968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0e103-18b9-4104-83d1-9ad4560fa1b1 service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:05.701254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0e103-18b9-4104-83d1-9ad4560fa1b1 service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:05.701550 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0e103-18b9-4104-83d1-9ad4560fa1b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:05.711640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0e103-18b9-4104-83d1-9ad4560fa1b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:05.713266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0e103-18b9-4104-83d1-9ad4560fa1b1 service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:05.730328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1214/2445] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:14:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:14:14.588688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e40d6-c3a4-49ef-8b2f-d06cb0999724 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:14.589163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e40d6-c3a4-49ef-8b2f-d06cb0999724 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:14.589276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e40d6-c3a4-49ef-8b2f-d06cb0999724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:14.596157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:14.607684 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-906e40d6-c3a4-49ef-8b2f-d06cb0999724 None None] MFA Rules not processed for user `1042fb5384e14cd2bc7090f09db977c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:14.608889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-906e40d6-c3a4-49ef-8b2f-d06cb0999724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:14.616160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1232/2446] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:15.718393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=75 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:17.171658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befffd65-ec75-46a5-a0ba-225679809e22 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:17.172206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befffd65-ec75-46a5-a0ba-225679809e22 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:17.172206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befffd65-ec75-46a5-a0ba-225679809e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:17.192759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-befffd65-ec75-46a5-a0ba-225679809e22 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:17.192759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-befffd65-ec75-46a5-a0ba-225679809e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:17.210848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1215/2447] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Jun 26 03:14:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:17.225114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e40d6-c3a4-49ef-8b2f-d06cb0999724 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:17.241290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906e40d6-c3a4-49ef-8b2f-d06cb0999724 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:17.247195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fc5fa-f3ea-44b9-854a-1ff10d5b63fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:17.247932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fc5fa-f3ea-44b9-854a-1ff10d5b63fb service swift] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:17.248307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fc5fa-f3ea-44b9-854a-1ff10d5b63fb service swift] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:17.248808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7fc5fa-f3ea-44b9-854a-1ff10d5b63fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:17.258538 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7fc5fa-f3ea-44b9-854a-1ff10d5b63fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:17.260158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7fc5fa-f3ea-44b9-854a-1ff10d5b63fb service swift] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:17.269385 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1233/2448] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:14:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:14:22.503057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befffd65-ec75-46a5-a0ba-225679809e22 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:22.559858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befffd65-ec75-46a5-a0ba-225679809e22 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:22.563647 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb6c88-7a48-46ee-b306-3318c85d43e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:22.564398 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb6c88-7a48-46ee-b306-3318c85d43e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:22.564759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb6c88-7a48-46ee-b306-3318c85d43e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:22.565107 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bb6c88-7a48-46ee-b306-3318c85d43e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:22.567205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-87bb6c88-7a48-46ee-b306-3318c85d43e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:22.671352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb6c88-7a48-46ee-b306-3318c85d43e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:22.672868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bb6c88-7a48-46ee-b306-3318c85d43e9 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:22.683239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1216/2449] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:26.159196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fc5fa-f3ea-44b9-854a-1ff10d5b63fb service swift] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:26.160726 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ea7fc5fa-f3ea-44b9-854a-1ff10d5b63fb service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:26.178843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7fc5fa-f3ea-44b9-854a-1ff10d5b63fb service swift] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:26.186727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:26.187852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:26.188358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:26.188836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] PATH_INFO: `/v3/users/95793ccb15844069890363d983f1680d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:26.190328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95793ccb15844069890363d983f1680d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:26.192491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:26.258304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95793ccb15844069890363d983f1680d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:26.259305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95793ccb15844069890363d983f1680d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:26.269092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95793ccb15844069890363d983f1680d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:26.293769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95793ccb15844069890363d983f1680d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:26.294426 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-94409d69-8861-4fe7-81a1-fc0f47782a19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95793ccb15844069890363d983f1680d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:26.319647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1234/2450] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:14:26 2022] DELETE /identity/v3/users/95793ccb15844069890363d983f1680d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:26.505680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb6c88-7a48-46ee-b306-3318c85d43e9 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:26.522795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bb6c88-7a48-46ee-b306-3318c85d43e9 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:26.606492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:26.607352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:26.607639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:26.607911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] PATH_INFO: `/v3/projects/ea86edd65ecb4972ba8bb33b34387432` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:26.609093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea86edd65ecb4972ba8bb33b34387432)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:26.611029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:26.662058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:26.665362 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea86edd65ecb4972ba8bb33b34387432', 'request_id': 'req-b20a5222-41af-4008-af0d-50c481875d3a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:26.673099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea86edd65ecb4972ba8bb33b34387432', 'request_id': 'req-b20a5222-41af-4008-af0d-50c481875d3a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:26.679008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea86edd65ecb4972ba8bb33b34387432', 'request_id': 'req-b20a5222-41af-4008-af0d-50c481875d3a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:26.689984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] The token cache is being invalidate because project ea86edd65ecb4972ba8bb33b34387432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:14:26.690373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:26.691089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b20a5222-41af-4008-af0d-50c481875d3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:26.692921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1217/2451] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:14:26 2022] DELETE /identity/v3/projects/ea86edd65ecb4972ba8bb33b34387432 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:26.698278 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dece8-e5f4-4737-8917-1b69fa358e72 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:26.698847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dece8-e5f4-4737-8917-1b69fa358e72 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:26.699128 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dece8-e5f4-4737-8917-1b69fa358e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:26.736683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f97dece8-e5f4-4737-8917-1b69fa358e72 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:26.737970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f97dece8-e5f4-4737-8917-1b69fa358e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:26.803623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1235/2452] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:14:26.811369 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6622b88-6304-47b0-9cf4-13d23c9c74af None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:26.811661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6622b88-6304-47b0-9cf4-13d23c9c74af None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:26.811941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6622b88-6304-47b0-9cf4-13d23c9c74af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:26.830951 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c6622b88-6304-47b0-9cf4-13d23c9c74af None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:26.832219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c6622b88-6304-47b0-9cf4-13d23c9c74af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:26.841021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1218/2453] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:14:26.845256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dece8-e5f4-4737-8917-1b69fa358e72 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:26.856783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dece8-e5f4-4737-8917-1b69fa358e72 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:26.861008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eff7ed-40f6-407c-ae04-448772bb399d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:26.861673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eff7ed-40f6-407c-ae04-448772bb399d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:26.861953 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eff7ed-40f6-407c-ae04-448772bb399d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:26.862263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eff7ed-40f6-407c-ae04-448772bb399d admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:26.863160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eff7ed-40f6-407c-ae04-448772bb399d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:14:26.863495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eff7ed-40f6-407c-ae04-448772bb399d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:26.865159 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eff7ed-40f6-407c-ae04-448772bb399d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:26.878673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1236/2454] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:14:26.883351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6622b88-6304-47b0-9cf4-13d23c9c74af None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:26.900463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6622b88-6304-47b0-9cf4-13d23c9c74af None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:26.905684 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decf947-f8bd-4acb-9f64-e01650f04e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:26.906415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decf947-f8bd-4acb-9f64-e01650f04e06 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:26.906733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decf947-f8bd-4acb-9f64-e01650f04e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:26.906970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8decf947-f8bd-4acb-9f64-e01650f04e06 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:26.907966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8decf947-f8bd-4acb-9f64-e01650f04e06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:26.909821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8decf947-f8bd-4acb-9f64-e01650f04e06 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:26.941186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1219/2455] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:26 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:26.945444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eff7ed-40f6-407c-ae04-448772bb399d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:26.956104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eff7ed-40f6-407c-ae04-448772bb399d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:27.000749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:27.028368 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96486a1-ead8-4ecc-8fa6-7dbcbaecf13e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:27.029075 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96486a1-ead8-4ecc-8fa6-7dbcbaecf13e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:27.029387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96486a1-ead8-4ecc-8fa6-7dbcbaecf13e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:27.029668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96486a1-ead8-4ecc-8fa6-7dbcbaecf13e admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:27.030424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96486a1-ead8-4ecc-8fa6-7dbcbaecf13e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:27.032402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96486a1-ead8-4ecc-8fa6-7dbcbaecf13e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:27.054807 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1237/2456] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:26 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:27.059258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decf947-f8bd-4acb-9f64-e01650f04e06 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:27.074574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8decf947-f8bd-4acb-9f64-e01650f04e06 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:27.079784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a8fa4-6c25-4f60-9067-1f689200e670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:27.080465 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a8fa4-6c25-4f60-9067-1f689200e670 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:27.080768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a8fa4-6c25-4f60-9067-1f689200e670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:27.081047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580a8fa4-6c25-4f60-9067-1f689200e670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:27.081782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a8fa4-6c25-4f60-9067-1f689200e670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:27.083444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580a8fa4-6c25-4f60-9067-1f689200e670 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:27.095454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1220/2457] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:27.099470 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96486a1-ead8-4ecc-8fa6-7dbcbaecf13e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:27.109845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96486a1-ead8-4ecc-8fa6-7dbcbaecf13e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:27.114490 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7083f7-ca7e-483a-b64e-82b251341bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:27.115149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7083f7-ca7e-483a-b64e-82b251341bff admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:27.115432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7083f7-ca7e-483a-b64e-82b251341bff admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:27.115706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7083f7-ca7e-483a-b64e-82b251341bff admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:27.116562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7083f7-ca7e-483a-b64e-82b251341bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:27.118508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7083f7-ca7e-483a-b64e-82b251341bff admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:27.130140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1238/2458] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:27.134810 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a8fa4-6c25-4f60-9067-1f689200e670 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:27.146384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580a8fa4-6c25-4f60-9067-1f689200e670 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:27.170097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8653f7-3a7d-481d-8a8a-4b7c9e1490bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:27.171274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8653f7-3a7d-481d-8a8a-4b7c9e1490bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:27.171878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8653f7-3a7d-481d-8a8a-4b7c9e1490bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:27.172582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8653f7-3a7d-481d-8a8a-4b7c9e1490bd admin admin] PATH_INFO: `/v3/projects/4effe04666a64c43b13c5c173cce4137/users/525f63b6795f4be88082e80b7c7dd172/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:27.201959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8653f7-3a7d-481d-8a8a-4b7c9e1490bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4effe04666a64c43b13c5c173cce4137, user_id=525f63b6795f4be88082e80b7c7dd172, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:27.205216 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8653f7-3a7d-481d-8a8a-4b7c9e1490bd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:27.222916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1221/2459] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:27 2022] PUT /identity/v3/projects/4effe04666a64c43b13c5c173cce4137/users/525f63b6795f4be88082e80b7c7dd172/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:27.234578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ef9fd-6ec8-4d85-b325-7d1b7f7e28a4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:27.235178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ef9fd-6ec8-4d85-b325-7d1b7f7e28a4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:27.235284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ef9fd-6ec8-4d85-b325-7d1b7f7e28a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:27.266465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4e4ef9fd-6ec8-4d85-b325-7d1b7f7e28a4 None None] MFA Rules not processed for user `525f63b6795f4be88082e80b7c7dd172`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:27.267779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4e4ef9fd-6ec8-4d85-b325-7d1b7f7e28a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:27.338574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1239/2460] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:27.349429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8653f7-3a7d-481d-8a8a-4b7c9e1490bd admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:27.351225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fd8653f7-3a7d-481d-8a8a-4b7c9e1490bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:27.427668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8653f7-3a7d-481d-8a8a-4b7c9e1490bd admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:27.433303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c2c0ba-7da2-4a2e-9fe9-57baedff216a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:27.433958 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2c0ba-7da2-4a2e-9fe9-57baedff216a service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:27.434274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2c0ba-7da2-4a2e-9fe9-57baedff216a service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:27.434557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c2c0ba-7da2-4a2e-9fe9-57baedff216a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:27.442756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2c0ba-7da2-4a2e-9fe9-57baedff216a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:27.444484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c2c0ba-7da2-4a2e-9fe9-57baedff216a service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:27.453989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1222/2461] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:37.333679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:37.449893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=4,keystone:SELECT=68,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:37.823556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ef9fd-6ec8-4d85-b325-7d1b7f7e28a4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:37.883844 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4ef9fd-6ec8-4d85-b325-7d1b7f7e28a4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:37.888872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:37.889740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:37.890034 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:37.890323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] PATH_INFO: `/v3/users/525f63b6795f4be88082e80b7c7dd172` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:37.891500 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] RBAC: Authorizing `identity:delete_user(user_id=525f63b6795f4be88082e80b7c7dd172)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:37.893406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:37.938230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '525f63b6795f4be88082e80b7c7dd172'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:37.938877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '525f63b6795f4be88082e80b7c7dd172'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:37.945635 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '525f63b6795f4be88082e80b7c7dd172'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:37.953690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '525f63b6795f4be88082e80b7c7dd172'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:37.954434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '525f63b6795f4be88082e80b7c7dd172'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:37.971751 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1240/2462] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:14:37 2022] DELETE /identity/v3/users/525f63b6795f4be88082e80b7c7dd172 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:37.976191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1977a9-7be4-4e11-a51c-fe332e22867d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:37.976479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1977a9-7be4-4e11-a51c-fe332e22867d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:37.976754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1977a9-7be4-4e11-a51c-fe332e22867d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:37.993334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1a1977a9-7be4-4e11-a51c-fe332e22867d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:37.994569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1a1977a9-7be4-4e11-a51c-fe332e22867d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:38.054021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1223/2463] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:38.064102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:38.065493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:38.124811 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004754a-8f5c-46b9-bb4d-0b33efbaf70b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:38.128768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2c638-b505-4da7-ba19-13a5a2aed734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:38.129787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2c638-b505-4da7-ba19-13a5a2aed734 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.130176 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2c638-b505-4da7-ba19-13a5a2aed734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.130527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2c638-b505-4da7-ba19-13a5a2aed734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.142967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2c638-b505-4da7-ba19-13a5a2aed734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:38.144320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2c638-b505-4da7-ba19-13a5a2aed734 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:38.156453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1241/2464] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:38.416988 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1977a9-7be4-4e11-a51c-fe332e22867d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:38.418919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1a1977a9-7be4-4e11-a51c-fe332e22867d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:38.434308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1977a9-7be4-4e11-a51c-fe332e22867d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:38.438962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:38.439831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.440114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.440389 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] PATH_INFO: `/v3/projects/4effe04666a64c43b13c5c173cce4137` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.441563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4effe04666a64c43b13c5c173cce4137)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:38.443500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:38.489607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4effe04666a64c43b13c5c173cce4137', 'request_id': 'req-6f24bc83-950f-4c70-b355-6e7fe09b193a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:38.497859 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4effe04666a64c43b13c5c173cce4137', 'request_id': 'req-6f24bc83-950f-4c70-b355-6e7fe09b193a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:38.503277 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4effe04666a64c43b13c5c173cce4137', 'request_id': 'req-6f24bc83-950f-4c70-b355-6e7fe09b193a'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:38.513996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] The token cache is being invalidate because project 4effe04666a64c43b13c5c173cce4137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:14:38.514309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:38.514925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f24bc83-950f-4c70-b355-6e7fe09b193a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:38.516424 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1224/2465] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:14:38 2022] DELETE /identity/v3/projects/4effe04666a64c43b13c5c173cce4137 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:38.521616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e93ae-f7cb-4ac2-8a3b-6458193a5a6f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.522151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e93ae-f7cb-4ac2-8a3b-6458193a5a6f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.522275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43e93ae-f7cb-4ac2-8a3b-6458193a5a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.539217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d43e93ae-f7cb-4ac2-8a3b-6458193a5a6f None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:38.540405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d43e93ae-f7cb-4ac2-8a3b-6458193a5a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:38.557069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:38.601240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1242/2466] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:38.609561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b9e08-0063-4016-947c-ce9bcbcc7fa5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.609855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b9e08-0063-4016-947c-ce9bcbcc7fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.610179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25b9e08-0063-4016-947c-ce9bcbcc7fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.629387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-e25b9e08-0063-4016-947c-ce9bcbcc7fa5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:38.630624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-e25b9e08-0063-4016-947c-ce9bcbcc7fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:38.638517 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1225/2467] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:38.642643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e93ae-f7cb-4ac2-8a3b-6458193a5a6f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:38.652347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43e93ae-f7cb-4ac2-8a3b-6458193a5a6f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:38.656852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f393893-090b-442a-9f4b-82a2adace006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:38.657499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f393893-090b-442a-9f4b-82a2adace006 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.657781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f393893-090b-442a-9f4b-82a2adace006 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.658082 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f393893-090b-442a-9f4b-82a2adace006 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.658808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f393893-090b-442a-9f4b-82a2adace006 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:14:38.659139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f393893-090b-442a-9f4b-82a2adace006 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:38.660770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f393893-090b-442a-9f4b-82a2adace006 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:38.672593 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1243/2468] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:14:38.676645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b9e08-0063-4016-947c-ce9bcbcc7fa5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:38.686275 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25b9e08-0063-4016-947c-ce9bcbcc7fa5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:38.691015 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b803456-e2e6-4dbd-9401-9d8a82d83380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:38.691691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803456-e2e6-4dbd-9401-9d8a82d83380 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.691968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803456-e2e6-4dbd-9401-9d8a82d83380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.692240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b803456-e2e6-4dbd-9401-9d8a82d83380 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.692921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b803456-e2e6-4dbd-9401-9d8a82d83380 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:38.694849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b803456-e2e6-4dbd-9401-9d8a82d83380 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:38.719884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1226/2469] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:38 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:38.723941 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f393893-090b-442a-9f4b-82a2adace006 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:38.734672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f393893-090b-442a-9f4b-82a2adace006 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:38.789448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fb696-0d9e-4b7d-a755-43053b3a175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:38.790131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fb696-0d9e-4b7d-a755-43053b3a175b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.790413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fb696-0d9e-4b7d-a755-43053b3a175b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.790686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73fb696-0d9e-4b7d-a755-43053b3a175b admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.791355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fb696-0d9e-4b7d-a755-43053b3a175b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:38.793874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73fb696-0d9e-4b7d-a755-43053b3a175b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:38.812471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1244/2470] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:38 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:14:38.816461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b803456-e2e6-4dbd-9401-9d8a82d83380 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:38.826759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b803456-e2e6-4dbd-9401-9d8a82d83380 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:38.831289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94afb85-6d71-47b7-9acd-bc02a5f390cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:38.831933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94afb85-6d71-47b7-9acd-bc02a5f390cc admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.832213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94afb85-6d71-47b7-9acd-bc02a5f390cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.832484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94afb85-6d71-47b7-9acd-bc02a5f390cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.833203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94afb85-6d71-47b7-9acd-bc02a5f390cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:38.834822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94afb85-6d71-47b7-9acd-bc02a5f390cc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:38.845840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1227/2471] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:14:38.849928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fb696-0d9e-4b7d-a755-43053b3a175b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:38.860948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73fb696-0d9e-4b7d-a755-43053b3a175b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:38.865541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb603f-1540-4e6f-83c4-5ebe21c41c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:38.866236 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb603f-1540-4e6f-83c4-5ebe21c41c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.866519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb603f-1540-4e6f-83c4-5ebe21c41c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.866791 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb603f-1540-4e6f-83c4-5ebe21c41c39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.867511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb603f-1540-4e6f-83c4-5ebe21c41c39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:38.869093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaeb603f-1540-4e6f-83c4-5ebe21c41c39 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:38.879763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1245/2472] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:38.883846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94afb85-6d71-47b7-9acd-bc02a5f390cc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:38.893654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94afb85-6d71-47b7-9acd-bc02a5f390cc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:38.898205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc59e0-d1e9-4e3c-b3bd-345cc48dcd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:38.899069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc59e0-d1e9-4e3c-b3bd-345cc48dcd83 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.899347 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc59e0-d1e9-4e3c-b3bd-345cc48dcd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.899620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc59e0-d1e9-4e3c-b3bd-345cc48dcd83 admin admin] PATH_INFO: `/v3/projects/e638c1a29b7c4b298a0f6aafd1b280f0/users/1e2ce0dbb7704a2bb7f3f49e84478370/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.914328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc59e0-d1e9-4e3c-b3bd-345cc48dcd83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e638c1a29b7c4b298a0f6aafd1b280f0, user_id=1e2ce0dbb7704a2bb7f3f49e84478370, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:38.916884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc59e0-d1e9-4e3c-b3bd-345cc48dcd83 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:38.930377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1228/2473] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:38 2022] PUT /identity/v3/projects/e638c1a29b7c4b298a0f6aafd1b280f0/users/1e2ce0dbb7704a2bb7f3f49e84478370/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:38.941701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb819ea1-3e8a-4cca-a95b-61c175d221e6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:38.941995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb819ea1-3e8a-4cca-a95b-61c175d221e6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:38.942309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb819ea1-3e8a-4cca-a95b-61c175d221e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:38.968428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-cb819ea1-3e8a-4cca-a95b-61c175d221e6 None None] MFA Rules not processed for user `1e2ce0dbb7704a2bb7f3f49e84478370`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:38.969636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-cb819ea1-3e8a-4cca-a95b-61c175d221e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:39.040221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1246/2474] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:38 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:39.054431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc59e0-d1e9-4e3c-b3bd-345cc48dcd83 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:39.056119 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d0fc59e0-d1e9-4e3c-b3bd-345cc48dcd83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:39.077498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:39.128710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc59e0-d1e9-4e3c-b3bd-345cc48dcd83 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:39.135270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2f762d-056b-4899-ac04-6b5295c1d086 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:39.135928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f762d-056b-4899-ac04-6b5295c1d086 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:39.136703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f762d-056b-4899-ac04-6b5295c1d086 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:39.137317 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2f762d-056b-4899-ac04-6b5295c1d086 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:39.145991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f762d-056b-4899-ac04-6b5295c1d086 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:39.147579 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2f762d-056b-4899-ac04-6b5295c1d086 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:39.156873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1229/2475] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:39.376293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb819ea1-3e8a-4cca-a95b-61c175d221e6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:39.395720 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:39.440709 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb819ea1-3e8a-4cca-a95b-61c175d221e6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:39.445135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:39.446070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:39.446371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:39.446645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] PATH_INFO: `/v3/users/1e2ce0dbb7704a2bb7f3f49e84478370` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:39.447854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e2ce0dbb7704a2bb7f3f49e84478370)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:39.450263 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:39.501259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e2ce0dbb7704a2bb7f3f49e84478370'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:39.502342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e2ce0dbb7704a2bb7f3f49e84478370'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:39.510905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e2ce0dbb7704a2bb7f3f49e84478370'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:39.519629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e2ce0dbb7704a2bb7f3f49e84478370'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:39.520298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e2ce0dbb7704a2bb7f3f49e84478370'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:39.541323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1247/2476] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:14:39 2022] DELETE /identity/v3/users/1e2ce0dbb7704a2bb7f3f49e84478370 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:39.545726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05441d3-5e94-4714-b8e3-0acc4538d858 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:39.546224 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05441d3-5e94-4714-b8e3-0acc4538d858 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:39.546448 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05441d3-5e94-4714-b8e3-0acc4538d858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:39.569388 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a05441d3-5e94-4714-b8e3-0acc4538d858 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:39.570711 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a05441d3-5e94-4714-b8e3-0acc4538d858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:39.636191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1230/2477] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:39.648380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:39.649931 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:39.713499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dd6c25-535e-49b6-80b0-6a0f8528ce90 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:39.717345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00323e-ff79-4f43-af27-b6a77df45506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:39.718105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00323e-ff79-4f43-af27-b6a77df45506 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:39.719144 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00323e-ff79-4f43-af27-b6a77df45506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:39.719144 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00323e-ff79-4f43-af27-b6a77df45506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:39.731213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00323e-ff79-4f43-af27-b6a77df45506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:39.733795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00323e-ff79-4f43-af27-b6a77df45506 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:39.744464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1248/2478] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:40.081262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05441d3-5e94-4714-b8e3-0acc4538d858 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.082776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a05441d3-5e94-4714-b8e3-0acc4538d858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:40.094792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05441d3-5e94-4714-b8e3-0acc4538d858 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.099577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.100451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.100743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.101100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] PATH_INFO: `/v3/projects/e638c1a29b7c4b298a0f6aafd1b280f0` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.102359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e638c1a29b7c4b298a0f6aafd1b280f0)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:40.104509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:40.151370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e638c1a29b7c4b298a0f6aafd1b280f0', 'request_id': 'req-c420bc4c-6351-4e20-804b-58c205ca87cd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:40.159012 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e638c1a29b7c4b298a0f6aafd1b280f0', 'request_id': 'req-c420bc4c-6351-4e20-804b-58c205ca87cd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:40.164796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e638c1a29b7c4b298a0f6aafd1b280f0', 'request_id': 'req-c420bc4c-6351-4e20-804b-58c205ca87cd'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:40.176141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] The token cache is being invalidate because project e638c1a29b7c4b298a0f6aafd1b280f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:14:40.176432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:40.177057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-c420bc4c-6351-4e20-804b-58c205ca87cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:40.178608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1231/2479] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:14:40 2022] DELETE /identity/v3/projects/e638c1a29b7c4b298a0f6aafd1b280f0 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:40.183429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e2ea7-9372-4892-b409-e346d3a5f8f5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.183721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e2ea7-9372-4892-b409-e346d3a5f8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.184644 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243e2ea7-9372-4892-b409-e346d3a5f8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.202753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-243e2ea7-9372-4892-b409-e346d3a5f8f5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:40.204078 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-243e2ea7-9372-4892-b409-e346d3a5f8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:40.238325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:40.265738 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1249/2480] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:40.273543 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b331be-c690-4fc7-a662-9c07c3802903 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.273829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b331be-c690-4fc7-a662-9c07c3802903 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.274135 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b331be-c690-4fc7-a662-9c07c3802903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.298421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-83b331be-c690-4fc7-a662-9c07c3802903 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:40.299821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-83b331be-c690-4fc7-a662-9c07c3802903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:40.308065 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1232/2481] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:40.312335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e2ea7-9372-4892-b409-e346d3a5f8f5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.321803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243e2ea7-9372-4892-b409-e346d3a5f8f5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.326369 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaee56f-ef4a-45f6-a133-5955cc09ed0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.327023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaee56f-ef4a-45f6-a133-5955cc09ed0a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.327328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaee56f-ef4a-45f6-a133-5955cc09ed0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.327603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaee56f-ef4a-45f6-a133-5955cc09ed0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.328777 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaee56f-ef4a-45f6-a133-5955cc09ed0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:14:40.328777 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaee56f-ef4a-45f6-a133-5955cc09ed0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:40.330209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaee56f-ef4a-45f6-a133-5955cc09ed0a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:40.343846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1250/2482] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:14:40.346547 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b331be-c690-4fc7-a662-9c07c3802903 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.355829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b331be-c690-4fc7-a662-9c07c3802903 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.361880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b525f-4818-45db-b052-d1a0abc71d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.362571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b525f-4818-45db-b052-d1a0abc71d67 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.362853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b525f-4818-45db-b052-d1a0abc71d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.363125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b525f-4818-45db-b052-d1a0abc71d67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.363817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b525f-4818-45db-b052-d1a0abc71d67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:40.365700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b525f-4818-45db-b052-d1a0abc71d67 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:40.387373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:40.395715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1233/2483] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:40 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:40.399883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaee56f-ef4a-45f6-a133-5955cc09ed0a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.410610 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaee56f-ef4a-45f6-a133-5955cc09ed0a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.472241 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf978fe-55ca-41a8-8488-865fec8e1bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.472881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf978fe-55ca-41a8-8488-865fec8e1bde admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.473193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf978fe-55ca-41a8-8488-865fec8e1bde admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.473469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf978fe-55ca-41a8-8488-865fec8e1bde admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.474239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf978fe-55ca-41a8-8488-865fec8e1bde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:40.477477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf978fe-55ca-41a8-8488-865fec8e1bde admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:40.501614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1251/2484] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:40 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:14:40.503381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b525f-4818-45db-b052-d1a0abc71d67 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.513236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b525f-4818-45db-b052-d1a0abc71d67 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.517792 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33cc21-4f6f-49e4-81bb-df336a68ecae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.518480 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33cc21-4f6f-49e4-81bb-df336a68ecae admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.518761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33cc21-4f6f-49e4-81bb-df336a68ecae admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.519032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33cc21-4f6f-49e4-81bb-df336a68ecae admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.519751 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33cc21-4f6f-49e4-81bb-df336a68ecae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:40.521341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33cc21-4f6f-49e4-81bb-df336a68ecae admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:40.532913 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1234/2485] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:40.537068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf978fe-55ca-41a8-8488-865fec8e1bde admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.547668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf978fe-55ca-41a8-8488-865fec8e1bde admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.553453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f3a33-ced7-4898-8492-7feab841f60a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.554175 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f3a33-ced7-4898-8492-7feab841f60a admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.554464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f3a33-ced7-4898-8492-7feab841f60a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.554745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f3a33-ced7-4898-8492-7feab841f60a admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.555476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f3a33-ced7-4898-8492-7feab841f60a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:40.557061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25f3a33-ced7-4898-8492-7feab841f60a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:40.567310 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1252/2486] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:40.571153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33cc21-4f6f-49e4-81bb-df336a68ecae admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.581074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33cc21-4f6f-49e4-81bb-df336a68ecae admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.585616 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3b9b2-035c-47cb-a396-848c962bf333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.586506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3b9b2-035c-47cb-a396-848c962bf333 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.586788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3b9b2-035c-47cb-a396-848c962bf333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.587061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e3b9b2-035c-47cb-a396-848c962bf333 admin admin] PATH_INFO: `/v3/projects/04a61e0d9c2b465dac95bdcb93e02ecc/users/dd685cf6b8244f34abb8904acb0c799b/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.601971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3b9b2-035c-47cb-a396-848c962bf333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a61e0d9c2b465dac95bdcb93e02ecc, user_id=dd685cf6b8244f34abb8904acb0c799b, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:40.604627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e3b9b2-035c-47cb-a396-848c962bf333 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:40.617793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1235/2487] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:40 2022] PUT /identity/v3/projects/04a61e0d9c2b465dac95bdcb93e02ecc/users/dd685cf6b8244f34abb8904acb0c799b/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:40.628111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeba48-1fe5-4378-9984-da52d6313a86 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.628510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeba48-1fe5-4378-9984-da52d6313a86 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.628881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aeba48-1fe5-4378-9984-da52d6313a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.656717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-04aeba48-1fe5-4378-9984-da52d6313a86 None None] MFA Rules not processed for user `dd685cf6b8244f34abb8904acb0c799b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:40.657918 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-04aeba48-1fe5-4378-9984-da52d6313a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:40.720605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1253/2488] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:40 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:40.725318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3b9b2-035c-47cb-a396-848c962bf333 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.785662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e3b9b2-035c-47cb-a396-848c962bf333 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.790287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07388a-caf3-4681-9cd2-c10995ffaf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.790933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07388a-caf3-4681-9cd2-c10995ffaf1a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.791220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07388a-caf3-4681-9cd2-c10995ffaf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.791498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07388a-caf3-4681-9cd2-c10995ffaf1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.792177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07388a-caf3-4681-9cd2-c10995ffaf1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:40.794151 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e07388a-caf3-4681-9cd2-c10995ffaf1a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:40.818431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1236/2489] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:40 2022] POST /identity/v3/projects => generated 405 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:40.822577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeba48-1fe5-4378-9984-da52d6313a86 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.833528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aeba48-1fe5-4378-9984-da52d6313a86 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.873771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:40.901300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee79253-54c1-4099-8114-95521e283e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.901974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee79253-54c1-4099-8114-95521e283e0f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.902297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee79253-54c1-4099-8114-95521e283e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.902563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee79253-54c1-4099-8114-95521e283e0f admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.903894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee79253-54c1-4099-8114-95521e283e0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:40.905820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee79253-54c1-4099-8114-95521e283e0f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:40.926831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1254/2490] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:40 2022] POST /identity/v3/users => generated 309 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:40.930770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07388a-caf3-4681-9cd2-c10995ffaf1a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.940944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07388a-caf3-4681-9cd2-c10995ffaf1a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.945631 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15f27c-ec3d-4e38-8c7f-efab33c6307c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.946337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15f27c-ec3d-4e38-8c7f-efab33c6307c admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.946624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15f27c-ec3d-4e38-8c7f-efab33c6307c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.946902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15f27c-ec3d-4e38-8c7f-efab33c6307c admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.947632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15f27c-ec3d-4e38-8c7f-efab33c6307c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:40.949222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15f27c-ec3d-4e38-8c7f-efab33c6307c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:40.961232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1237/2491] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:14:40.965965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee79253-54c1-4099-8114-95521e283e0f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:40.975973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee79253-54c1-4099-8114-95521e283e0f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:40.980438 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e3afc-d45b-47b5-b1e7-8d31dc900393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:40.981338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e3afc-d45b-47b5-b1e7-8d31dc900393 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:40.981617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e3afc-d45b-47b5-b1e7-8d31dc900393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:40.981890 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e3afc-d45b-47b5-b1e7-8d31dc900393 admin admin] PATH_INFO: `/v3/projects/bfb8c72fb94a40678d3c8b3c845381a3/users/c7bcb8945d984502b0cbf87133740aa6/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:40.997763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e3afc-d45b-47b5-b1e7-8d31dc900393 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb8c72fb94a40678d3c8b3c845381a3, user_id=c7bcb8945d984502b0cbf87133740aa6, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:41.000398 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e3afc-d45b-47b5-b1e7-8d31dc900393 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:41.013913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1255/2492] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:40 2022] PUT /identity/v3/projects/bfb8c72fb94a40678d3c8b3c845381a3/users/c7bcb8945d984502b0cbf87133740aa6/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:14:41.017838 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15f27c-ec3d-4e38-8c7f-efab33c6307c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:41.027676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15f27c-ec3d-4e38-8c7f-efab33c6307c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:41.082493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae07e4d-1654-4d32-8864-5e4dd062ffe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:41.083153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae07e4d-1654-4d32-8864-5e4dd062ffe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:41.083438 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae07e4d-1654-4d32-8864-5e4dd062ffe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:41.083715 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae07e4d-1654-4d32-8864-5e4dd062ffe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:41.084436 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae07e4d-1654-4d32-8864-5e4dd062ffe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:41.086057 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae07e4d-1654-4d32-8864-5e4dd062ffe0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:41.095878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1238/2493] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:41.099926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e3afc-d45b-47b5-b1e7-8d31dc900393 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:41.109799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e3afc-d45b-47b5-b1e7-8d31dc900393 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:41.114281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069ab2a-9ca9-4396-9245-6c2a24f92e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:41.115137 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069ab2a-9ca9-4396-9245-6c2a24f92e34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:41.115411 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069ab2a-9ca9-4396-9245-6c2a24f92e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:41.115678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069ab2a-9ca9-4396-9245-6c2a24f92e34 admin admin] PATH_INFO: `/v3/projects/bfb8c72fb94a40678d3c8b3c845381a3/users/c7bcb8945d984502b0cbf87133740aa6/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:41.117576 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069ab2a-9ca9-4396-9245-6c2a24f92e34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfb8c72fb94a40678d3c8b3c845381a3, user_id=c7bcb8945d984502b0cbf87133740aa6, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:41.120192 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069ab2a-9ca9-4396-9245-6c2a24f92e34 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:41.131416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1256/2494] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:41 2022] PUT /identity/v3/projects/bfb8c72fb94a40678d3c8b3c845381a3/users/c7bcb8945d984502b0cbf87133740aa6/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:41.143376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cd45d-9083-41b3-9244-f0e755668146 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:41.143691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cd45d-9083-41b3-9244-f0e755668146 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:41.143956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cd45d-9083-41b3-9244-f0e755668146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:41.169952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-583cd45d-9083-41b3-9244-f0e755668146 None None] MFA Rules not processed for user `c7bcb8945d984502b0cbf87133740aa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:41.171209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-583cd45d-9083-41b3-9244-f0e755668146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:41.195429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:41.239625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1239/2495] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:41 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:41.250534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069ab2a-9ca9-4396-9245-6c2a24f92e34 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:41.252212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b069ab2a-9ca9-4396-9245-6c2a24f92e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:41.306554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069ab2a-9ca9-4396-9245-6c2a24f92e34 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:41.310570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b4f75c-069b-4620-8be1-9b3b45df5b43 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:41.311191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4f75c-069b-4620-8be1-9b3b45df5b43 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:41.311472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4f75c-069b-4620-8be1-9b3b45df5b43 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:41.311743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b4f75c-069b-4620-8be1-9b3b45df5b43 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:41.320302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4f75c-069b-4620-8be1-9b3b45df5b43 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:41.321706 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b4f75c-069b-4620-8be1-9b3b45df5b43 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:41.331384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1257/2496] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:41 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:14:41.474598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cd45d-9083-41b3-9244-f0e755668146 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:41.536059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cd45d-9083-41b3-9244-f0e755668146 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:41.540553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:41.541423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:41.541707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:41.541984 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] PATH_INFO: `/v3/users/dd685cf6b8244f34abb8904acb0c799b` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:41.543205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd685cf6b8244f34abb8904acb0c799b)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:41.545653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:41.593888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd685cf6b8244f34abb8904acb0c799b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:41.594575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd685cf6b8244f34abb8904acb0c799b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:41.595189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd685cf6b8244f34abb8904acb0c799b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:41.607455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd685cf6b8244f34abb8904acb0c799b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:41.618790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd685cf6b8244f34abb8904acb0c799b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:41.634114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1240/2497] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:14:41 2022] DELETE /identity/v3/users/dd685cf6b8244f34abb8904acb0c799b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:41.638865 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d885b6-601c-45a0-8786-94af22d7a475 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:41.639370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d885b6-601c-45a0-8786-94af22d7a475 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:41.639484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d885b6-601c-45a0-8786-94af22d7a475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:41.656897 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-57d885b6-601c-45a0-8786-94af22d7a475 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:41.658145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-57d885b6-601c-45a0-8786-94af22d7a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:41.722885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1258/2498] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:41.729594 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:41.731531 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:41.778242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:41.794625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637b14b-307a-42db-9243-ce5642d28b33 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:41.798323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec7229-879b-4b2b-8a50-5e85bd4d24d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:41.798968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec7229-879b-4b2b-8a50-5e85bd4d24d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:41.799253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec7229-879b-4b2b-8a50-5e85bd4d24d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:41.799532 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec7229-879b-4b2b-8a50-5e85bd4d24d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:41.808604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec7229-879b-4b2b-8a50-5e85bd4d24d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:41.810079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec7229-879b-4b2b-8a50-5e85bd4d24d4 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:41.819789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1241/2499] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:42.080953 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d885b6-601c-45a0-8786-94af22d7a475 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:42.082536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-57d885b6-601c-45a0-8786-94af22d7a475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:42.095887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d885b6-601c-45a0-8786-94af22d7a475 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:42.100514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:42.101377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:42.101658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:42.101929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] PATH_INFO: `/v3/projects/04a61e0d9c2b465dac95bdcb93e02ecc` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:42.103171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a61e0d9c2b465dac95bdcb93e02ecc)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:42.105053 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:42.135242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:42.152506 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a61e0d9c2b465dac95bdcb93e02ecc', 'request_id': 'req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.158662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a61e0d9c2b465dac95bdcb93e02ecc', 'request_id': 'req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.165510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a61e0d9c2b465dac95bdcb93e02ecc', 'request_id': 'req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.176264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] The token cache is being invalidate because project 04a61e0d9c2b465dac95bdcb93e02ecc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:14:42.176548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.177162 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.178849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1259/2500] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:14:42 2022] DELETE /identity/v3/projects/04a61e0d9c2b465dac95bdcb93e02ecc => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:42.182760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec7229-879b-4b2b-8a50-5e85bd4d24d4 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:42.192234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec7229-879b-4b2b-8a50-5e85bd4d24d4 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:42.248159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:42.249027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:42.249292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:42.249563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] PATH_INFO: `/v3/users/c7bcb8945d984502b0cbf87133740aa6` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:42.250751 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7bcb8945d984502b0cbf87133740aa6)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:42.252644 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:42.297169 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7bcb8945d984502b0cbf87133740aa6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.297796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7bcb8945d984502b0cbf87133740aa6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.298401 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7bcb8945d984502b0cbf87133740aa6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.309959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7bcb8945d984502b0cbf87133740aa6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.316821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-68d1614f-a64f-48f4-846a-77ae95cf89fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7bcb8945d984502b0cbf87133740aa6'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.329497 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1242/2501] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:14:42 2022] DELETE /identity/v3/users/c7bcb8945d984502b0cbf87133740aa6 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:42.606203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:42.607617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:42.620776 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c5cfa-687c-49c1-a01a-ac1939a6a1ad admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:42.679773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:42.680642 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:42.680922 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:42.681192 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] PATH_INFO: `/v3/projects/bfb8c72fb94a40678d3c8b3c845381a3` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:42.682399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfb8c72fb94a40678d3c8b3c845381a3)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:42.684847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:42.730855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfb8c72fb94a40678d3c8b3c845381a3', 'request_id': 'req-32556d27-1c2c-4333-927e-850240b3e056'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.736824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfb8c72fb94a40678d3c8b3c845381a3', 'request_id': 'req-32556d27-1c2c-4333-927e-850240b3e056'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.744493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfb8c72fb94a40678d3c8b3c845381a3', 'request_id': 'req-32556d27-1c2c-4333-927e-850240b3e056'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.756412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] The token cache is being invalidate because project bfb8c72fb94a40678d3c8b3c845381a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:14:42.756697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.757333 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-32556d27-1c2c-4333-927e-850240b3e056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:42.759070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1260/2502] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:14:42 2022] DELETE /identity/v3/projects/bfb8c72fb94a40678d3c8b3c845381a3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:42.764487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18554ec3-9fbb-491a-90a2-5ec0cd3c2603 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:42.764996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18554ec3-9fbb-491a-90a2-5ec0cd3c2603 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:42.765098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18554ec3-9fbb-491a-90a2-5ec0cd3c2603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:42.782165 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-18554ec3-9fbb-491a-90a2-5ec0cd3c2603 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:42.783367 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-18554ec3-9fbb-491a-90a2-5ec0cd3c2603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:42.849037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1243/2503] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:42.850926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b9251-767b-40c5-8336-ab6743b2edb2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:42.851388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b9251-767b-40c5-8336-ab6743b2edb2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:42.851532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b9251-767b-40c5-8336-ab6743b2edb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:42.873970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-b63b9251-767b-40c5-8336-ab6743b2edb2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:42.875258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-b63b9251-767b-40c5-8336-ab6743b2edb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:42.883805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1261/2504] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:42.888114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18554ec3-9fbb-491a-90a2-5ec0cd3c2603 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:42.893709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:42.899361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18554ec3-9fbb-491a-90a2-5ec0cd3c2603 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:42.908117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4eb4b-ba48-4416-8c2a-1852fa15619e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:42.909054 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4eb4b-ba48-4416-8c2a-1852fa15619e admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:42.909403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4eb4b-ba48-4416-8c2a-1852fa15619e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:42.910067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e4eb4b-ba48-4416-8c2a-1852fa15619e admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:42.910879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4eb4b-ba48-4416-8c2a-1852fa15619e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:14:42.911284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4eb4b-ba48-4416-8c2a-1852fa15619e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:42.913258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e4eb4b-ba48-4416-8c2a-1852fa15619e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:42.929302 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1244/2505] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:14:42.930464 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b9251-767b-40c5-8336-ab6743b2edb2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:42.940498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b9251-767b-40c5-8336-ab6743b2edb2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:42.945127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec3a4d-2011-41ea-b05d-9d0b18acc32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:42.945787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec3a4d-2011-41ea-b05d-9d0b18acc32f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:42.946096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec3a4d-2011-41ea-b05d-9d0b18acc32f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:42.946379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eec3a4d-2011-41ea-b05d-9d0b18acc32f admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:42.947064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec3a4d-2011-41ea-b05d-9d0b18acc32f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:42.948991 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eec3a4d-2011-41ea-b05d-9d0b18acc32f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:42.978224 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1262/2506] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:42 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:42.979393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4eb4b-ba48-4416-8c2a-1852fa15619e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:42.989712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e4eb4b-ba48-4416-8c2a-1852fa15619e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.045967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c45372-6e51-4a98-84d2-27e08570d7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.046673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c45372-6e51-4a98-84d2-27e08570d7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.046960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c45372-6e51-4a98-84d2-27e08570d7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.047240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c45372-6e51-4a98-84d2-27e08570d7d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.047941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c45372-6e51-4a98-84d2-27e08570d7d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.049865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c45372-6e51-4a98-84d2-27e08570d7d2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.076420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1245/2507] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:42 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:14:43.077310 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec3a4d-2011-41ea-b05d-9d0b18acc32f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.088461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eec3a4d-2011-41ea-b05d-9d0b18acc32f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.093202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e562e-a0ba-4e35-83a1-21230e4e9342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.093859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e562e-a0ba-4e35-83a1-21230e4e9342 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.094182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e562e-a0ba-4e35-83a1-21230e4e9342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.094463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e562e-a0ba-4e35-83a1-21230e4e9342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.095178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e562e-a0ba-4e35-83a1-21230e4e9342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.096756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e562e-a0ba-4e35-83a1-21230e4e9342 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.107917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1263/2508] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:43.111929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c45372-6e51-4a98-84d2-27e08570d7d2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.122520 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c45372-6e51-4a98-84d2-27e08570d7d2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.127208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c120921d-28ad-4d95-bdec-c0fd0950c347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.128340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120921d-28ad-4d95-bdec-c0fd0950c347 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.128819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120921d-28ad-4d95-bdec-c0fd0950c347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.129551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c120921d-28ad-4d95-bdec-c0fd0950c347 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.130947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120921d-28ad-4d95-bdec-c0fd0950c347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.134198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c120921d-28ad-4d95-bdec-c0fd0950c347 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.145496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1246/2509] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:43.149748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e562e-a0ba-4e35-83a1-21230e4e9342 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.156799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:43.160508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e562e-a0ba-4e35-83a1-21230e4e9342 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.166689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64be7-9433-4516-942a-25e3011a4e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.167772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64be7-9433-4516-942a-25e3011a4e08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.168126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64be7-9433-4516-942a-25e3011a4e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.169211 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac64be7-9433-4516-942a-25e3011a4e08 admin admin] PATH_INFO: `/v3/projects/b98da6bd4a1b4876845fdb5a05fdc5c2/users/e551f1b20176492ba8bf3d227d73de5d/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.186732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac64be7-9433-4516-942a-25e3011a4e08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b98da6bd4a1b4876845fdb5a05fdc5c2, user_id=e551f1b20176492ba8bf3d227d73de5d, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.189370 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac64be7-9433-4516-942a-25e3011a4e08 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.203992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1264/2510] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:43 2022] PUT /identity/v3/projects/b98da6bd4a1b4876845fdb5a05fdc5c2/users/e551f1b20176492ba8bf3d227d73de5d/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:43.214238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc016982-d801-4308-b399-362bd5884d29 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.214536 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc016982-d801-4308-b399-362bd5884d29 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.214815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc016982-d801-4308-b399-362bd5884d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.243357 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-dc016982-d801-4308-b399-362bd5884d29 None None] MFA Rules not processed for user `e551f1b20176492ba8bf3d227d73de5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:43.244559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-dc016982-d801-4308-b399-362bd5884d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:43.314100 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1247/2511] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:43 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:43.318615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64be7-9433-4516-942a-25e3011a4e08 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.381416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac64be7-9433-4516-942a-25e3011a4e08 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.386153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d8305-e0da-433f-8c3f-2616caa23e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.386815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d8305-e0da-433f-8c3f-2616caa23e3e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.387133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d8305-e0da-433f-8c3f-2616caa23e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.387416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79d8305-e0da-433f-8c3f-2616caa23e3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.388106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d8305-e0da-433f-8c3f-2616caa23e3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.390177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79d8305-e0da-433f-8c3f-2616caa23e3e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.418709 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1265/2512] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:43 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:43.419852 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc016982-d801-4308-b399-362bd5884d29 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.430746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc016982-d801-4308-b399-362bd5884d29 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.483763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:43.490504 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e90a6b-7f99-4a1a-bd22-d78090a65495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.491303 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e90a6b-7f99-4a1a-bd22-d78090a65495 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.491823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e90a6b-7f99-4a1a-bd22-d78090a65495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.492322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e90a6b-7f99-4a1a-bd22-d78090a65495 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.493065 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e90a6b-7f99-4a1a-bd22-d78090a65495 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.495245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e90a6b-7f99-4a1a-bd22-d78090a65495 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.516282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1248/2513] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:43 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:43.520291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d8305-e0da-433f-8c3f-2616caa23e3e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.531022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79d8305-e0da-433f-8c3f-2616caa23e3e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.535549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352314a2-4dd1-4f60-9e19-192791df0b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.536203 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352314a2-4dd1-4f60-9e19-192791df0b3e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.536484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352314a2-4dd1-4f60-9e19-192791df0b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.536756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352314a2-4dd1-4f60-9e19-192791df0b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.537472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352314a2-4dd1-4f60-9e19-192791df0b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.539083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352314a2-4dd1-4f60-9e19-192791df0b3e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.550388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1266/2514] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:43.554562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e90a6b-7f99-4a1a-bd22-d78090a65495 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.566145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e90a6b-7f99-4a1a-bd22-d78090a65495 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.570929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc2994-ed6e-486f-b921-4c422cbe0e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.571818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc2994-ed6e-486f-b921-4c422cbe0e2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.572315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc2994-ed6e-486f-b921-4c422cbe0e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.572600 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc2994-ed6e-486f-b921-4c422cbe0e2d admin admin] PATH_INFO: `/v3/projects/37db65eb2fd04e5daedb9f3a7b59fd9a/users/908b0ae2c2a94dffbc1dcf873bfb66f2/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.588656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc2994-ed6e-486f-b921-4c422cbe0e2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=37db65eb2fd04e5daedb9f3a7b59fd9a, user_id=908b0ae2c2a94dffbc1dcf873bfb66f2, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.591435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc2994-ed6e-486f-b921-4c422cbe0e2d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.604088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1249/2515] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:43 2022] PUT /identity/v3/projects/37db65eb2fd04e5daedb9f3a7b59fd9a/users/908b0ae2c2a94dffbc1dcf873bfb66f2/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:14:43.609035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352314a2-4dd1-4f60-9e19-192791df0b3e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.620473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352314a2-4dd1-4f60-9e19-192791df0b3e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.680693 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a026b1-61e5-4a5e-aa09-3ab55ed69b80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.681363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a026b1-61e5-4a5e-aa09-3ab55ed69b80 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.681688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a026b1-61e5-4a5e-aa09-3ab55ed69b80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.682000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a026b1-61e5-4a5e-aa09-3ab55ed69b80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.682775 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a026b1-61e5-4a5e-aa09-3ab55ed69b80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.685116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a026b1-61e5-4a5e-aa09-3ab55ed69b80 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.698586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1267/2516] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:43.700789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc2994-ed6e-486f-b921-4c422cbe0e2d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.711145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc2994-ed6e-486f-b921-4c422cbe0e2d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.716414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e78196-1e0a-448c-b176-95a17aec294f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.717302 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e78196-1e0a-448c-b176-95a17aec294f admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.717574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e78196-1e0a-448c-b176-95a17aec294f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.717860 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e78196-1e0a-448c-b176-95a17aec294f admin admin] PATH_INFO: `/v3/projects/37db65eb2fd04e5daedb9f3a7b59fd9a/users/908b0ae2c2a94dffbc1dcf873bfb66f2/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.720245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e78196-1e0a-448c-b176-95a17aec294f admin admin] RBAC: Authorizing `identity:create_grant(project_id=37db65eb2fd04e5daedb9f3a7b59fd9a, user_id=908b0ae2c2a94dffbc1dcf873bfb66f2, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.722496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e78196-1e0a-448c-b176-95a17aec294f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.734368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1250/2517] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:43 2022] PUT /identity/v3/projects/37db65eb2fd04e5daedb9f3a7b59fd9a/users/908b0ae2c2a94dffbc1dcf873bfb66f2/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:14:43.746271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b64446-d3ad-48d1-ad80-a54e01afdda1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.747361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b64446-d3ad-48d1-ad80-a54e01afdda1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.747830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b64446-d3ad-48d1-ad80-a54e01afdda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.780339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-10b64446-d3ad-48d1-ad80-a54e01afdda1 None None] MFA Rules not processed for user `908b0ae2c2a94dffbc1dcf873bfb66f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:43.781739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-10b64446-d3ad-48d1-ad80-a54e01afdda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:43.823406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:43.851799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1268/2518] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:43 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:43.864034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e78196-1e0a-448c-b176-95a17aec294f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.865750 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c5e78196-1e0a-448c-b176-95a17aec294f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:43.921349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e78196-1e0a-448c-b176-95a17aec294f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:43.924845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d47206-cbdc-4501-a13f-107df9226300 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:43.925476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d47206-cbdc-4501-a13f-107df9226300 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:43.925758 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d47206-cbdc-4501-a13f-107df9226300 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:43.926033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d47206-cbdc-4501-a13f-107df9226300 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:43.935351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d47206-cbdc-4501-a13f-107df9226300 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:43.936790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d47206-cbdc-4501-a13f-107df9226300 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:43.945986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1251/2519] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:43 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:43.988035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b64446-d3ad-48d1-ad80-a54e01afdda1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:43.989410 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-10b64446-d3ad-48d1-ad80-a54e01afdda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:44.002537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b64446-d3ad-48d1-ad80-a54e01afdda1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:44.006511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6266276-ea77-4f9d-8f77-2e1f7ffa9bf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:44.007273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6266276-ea77-4f9d-8f77-2e1f7ffa9bf0 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:44.007528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6266276-ea77-4f9d-8f77-2e1f7ffa9bf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:44.007827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6266276-ea77-4f9d-8f77-2e1f7ffa9bf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:44.065637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6266276-ea77-4f9d-8f77-2e1f7ffa9bf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:44.067947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6266276-ea77-4f9d-8f77-2e1f7ffa9bf0 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:44.076928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1269/2520] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:43 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:47.349642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d47206-cbdc-4501-a13f-107df9226300 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:47.420186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d47206-cbdc-4501-a13f-107df9226300 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:47.424774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:47.425646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:47.425932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:47.426241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] PATH_INFO: `/v3/users/e551f1b20176492ba8bf3d227d73de5d` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:47.427486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=e551f1b20176492ba8bf3d227d73de5d)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:47.429403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:47.476718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e551f1b20176492ba8bf3d227d73de5d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:47.477379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e551f1b20176492ba8bf3d227d73de5d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:47.477992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e551f1b20176492ba8bf3d227d73de5d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:47.486007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:47.491121 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e551f1b20176492ba8bf3d227d73de5d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:47.499458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e551f1b20176492ba8bf3d227d73de5d'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:47.513336 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1252/2521] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:14:47 2022] DELETE /identity/v3/users/e551f1b20176492ba8bf3d227d73de5d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:47.518503 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3560036-7693-4429-970f-8582d0fde444 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:47.519077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3560036-7693-4429-970f-8582d0fde444 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:47.519198 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3560036-7693-4429-970f-8582d0fde444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:47.537755 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c3560036-7693-4429-970f-8582d0fde444 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:47.538982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c3560036-7693-4429-970f-8582d0fde444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:47.605377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1270/2522] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:47.615344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:47.617142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:47.686203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e20e48-fb13-4054-a705-bb5d9f4c54fd admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:47.686203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3da580-80b8-4fdd-821f-0295ac5dc48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:47.686203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3da580-80b8-4fdd-821f-0295ac5dc48d service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:47.686203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3da580-80b8-4fdd-821f-0295ac5dc48d service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:47.686541 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3da580-80b8-4fdd-821f-0295ac5dc48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:47.696561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3da580-80b8-4fdd-821f-0295ac5dc48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:47.697990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3da580-80b8-4fdd-821f-0295ac5dc48d service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:47.710720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1253/2523] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:14:47.973300 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3560036-7693-4429-970f-8582d0fde444 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:47.974742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c3560036-7693-4429-970f-8582d0fde444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:47.988385 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3560036-7693-4429-970f-8582d0fde444 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:47.992876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:47.994238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:47.994521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:47.994793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] PATH_INFO: `/v3/projects/b98da6bd4a1b4876845fdb5a05fdc5c2` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:47.996146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b98da6bd4a1b4876845fdb5a05fdc5c2)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:47.998521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:48.044395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b98da6bd4a1b4876845fdb5a05fdc5c2', 'request_id': 'req-f89a4e4f-e0e5-4140-b176-8474562fa507'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.049918 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b98da6bd4a1b4876845fdb5a05fdc5c2', 'request_id': 'req-f89a4e4f-e0e5-4140-b176-8474562fa507'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.057105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b98da6bd4a1b4876845fdb5a05fdc5c2', 'request_id': 'req-f89a4e4f-e0e5-4140-b176-8474562fa507'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.064620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:48.069673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] The token cache is being invalidate because project b98da6bd4a1b4876845fdb5a05fdc5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:14:48.070153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.071249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.073040 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1271/2524] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:14:47 2022] DELETE /identity/v3/projects/b98da6bd4a1b4876845fdb5a05fdc5c2 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:48.077627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3da580-80b8-4fdd-821f-0295ac5dc48d service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:48.087176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3da580-80b8-4fdd-821f-0295ac5dc48d service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:48.144873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:48.145725 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:48.146006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:48.146311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] PATH_INFO: `/v3/users/908b0ae2c2a94dffbc1dcf873bfb66f2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:48.147485 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=908b0ae2c2a94dffbc1dcf873bfb66f2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:48.149375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:48.193510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '908b0ae2c2a94dffbc1dcf873bfb66f2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.194176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '908b0ae2c2a94dffbc1dcf873bfb66f2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.194779 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '908b0ae2c2a94dffbc1dcf873bfb66f2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.206574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '908b0ae2c2a94dffbc1dcf873bfb66f2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.213613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-70fd4f58-f7a9-47df-b8e8-23510d99fef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '908b0ae2c2a94dffbc1dcf873bfb66f2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.218403 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:48.228306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1254/2525] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:14:48 2022] DELETE /identity/v3/users/908b0ae2c2a94dffbc1dcf873bfb66f2 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:48.497691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:48.499148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:48.514166 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a4e4f-e0e5-4140-b176-8474562fa507 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:48.571306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:48.572167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:48.572441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:48.572713 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] PATH_INFO: `/v3/projects/37db65eb2fd04e5daedb9f3a7b59fd9a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:48.573898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37db65eb2fd04e5daedb9f3a7b59fd9a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:48.575874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:48.620476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37db65eb2fd04e5daedb9f3a7b59fd9a', 'request_id': 'req-ef7ddc25-7ddf-4951-bc38-119e64717ea6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.626001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37db65eb2fd04e5daedb9f3a7b59fd9a', 'request_id': 'req-ef7ddc25-7ddf-4951-bc38-119e64717ea6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.633754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37db65eb2fd04e5daedb9f3a7b59fd9a', 'request_id': 'req-ef7ddc25-7ddf-4951-bc38-119e64717ea6'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.644527 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] The token cache is being invalidate because project 37db65eb2fd04e5daedb9f3a7b59fd9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:14:48.644824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.645437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-ef7ddc25-7ddf-4951-bc38-119e64717ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:48.647308 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1272/2526] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:14:48 2022] DELETE /identity/v3/projects/37db65eb2fd04e5daedb9f3a7b59fd9a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:48.652167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56332f1-2ddf-44dd-bc57-98587d120fbc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:48.653158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56332f1-2ddf-44dd-bc57-98587d120fbc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:48.653273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56332f1-2ddf-44dd-bc57-98587d120fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:48.671079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a56332f1-2ddf-44dd-bc57-98587d120fbc None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:48.672210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a56332f1-2ddf-44dd-bc57-98587d120fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:48.732833 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1255/2527] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:48.740675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa7aaf-d0be-41e1-b78c-7f8df5dc233b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:48.740974 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa7aaf-d0be-41e1-b78c-7f8df5dc233b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:48.741253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa7aaf-d0be-41e1-b78c-7f8df5dc233b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:48.759499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-6afa7aaf-d0be-41e1-b78c-7f8df5dc233b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:48.760708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6afa7aaf-d0be-41e1-b78c-7f8df5dc233b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:48.768563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1273/2528] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:48.772894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56332f1-2ddf-44dd-bc57-98587d120fbc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:48.783172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56332f1-2ddf-44dd-bc57-98587d120fbc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:48.787961 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afaffc-681f-4384-af9f-2c96b4ecddc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:48.788618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afaffc-681f-4384-af9f-2c96b4ecddc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:48.788908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afaffc-681f-4384-af9f-2c96b4ecddc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:48.789188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9afaffc-681f-4384-af9f-2c96b4ecddc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:48.789917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afaffc-681f-4384-af9f-2c96b4ecddc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:14:48.790287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afaffc-681f-4384-af9f-2c96b4ecddc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:48.791943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9afaffc-681f-4384-af9f-2c96b4ecddc8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:48.804231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1256/2529] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:14:48.808270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa7aaf-d0be-41e1-b78c-7f8df5dc233b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:48.817465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afa7aaf-d0be-41e1-b78c-7f8df5dc233b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:48.822127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354471-920a-46a0-ac81-573d581e226f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:48.822773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354471-920a-46a0-ac81-573d581e226f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:48.823058 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354471-920a-46a0-ac81-573d581e226f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:48.823360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354471-920a-46a0-ac81-573d581e226f admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:48.824048 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9354471-920a-46a0-ac81-573d581e226f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:48.825926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9354471-920a-46a0-ac81-573d581e226f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:48.851553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1274/2530] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:48 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:48.855601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afaffc-681f-4384-af9f-2c96b4ecddc8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:48.866163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9afaffc-681f-4384-af9f-2c96b4ecddc8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:48.921606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80860bc1-6572-4c8a-ae08-3c58d905b486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:48.922235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860bc1-6572-4c8a-ae08-3c58d905b486 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:48.922529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860bc1-6572-4c8a-ae08-3c58d905b486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:48.922807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80860bc1-6572-4c8a-ae08-3c58d905b486 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:48.923490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80860bc1-6572-4c8a-ae08-3c58d905b486 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:48.925398 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80860bc1-6572-4c8a-ae08-3c58d905b486 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:48.945653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1257/2531] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:48 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:48.949867 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354471-920a-46a0-ac81-573d581e226f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:48.960418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354471-920a-46a0-ac81-573d581e226f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:48.964892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9ffd4-f5ca-44ee-864d-5b7c1e3fca10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:48.965543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9ffd4-f5ca-44ee-864d-5b7c1e3fca10 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:48.966475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9ffd4-f5ca-44ee-864d-5b7c1e3fca10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:48.966761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9ffd4-f5ca-44ee-864d-5b7c1e3fca10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:48.967483 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9ffd4-f5ca-44ee-864d-5b7c1e3fca10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:48.969055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe9ffd4-f5ca-44ee-864d-5b7c1e3fca10 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:48.979605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1275/2532] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:48.983292 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80860bc1-6572-4c8a-ae08-3c58d905b486 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:48.993123 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80860bc1-6572-4c8a-ae08-3c58d905b486 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:48.997871 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df86404d-e426-42f3-bab7-c7ead48a2400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:48.998576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86404d-e426-42f3-bab7-c7ead48a2400 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:48.998865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86404d-e426-42f3-bab7-c7ead48a2400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:48.999145 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df86404d-e426-42f3-bab7-c7ead48a2400 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:48.999926 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86404d-e426-42f3-bab7-c7ead48a2400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:49.001508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df86404d-e426-42f3-bab7-c7ead48a2400 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:49.011928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1258/2533] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:49.016291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9ffd4-f5ca-44ee-864d-5b7c1e3fca10 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:49.026408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9ffd4-f5ca-44ee-864d-5b7c1e3fca10 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:49.031686 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6f305-c9d8-433f-bcf6-a3b0a8c36682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:49.032585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6f305-c9d8-433f-bcf6-a3b0a8c36682 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.032870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6f305-c9d8-433f-bcf6-a3b0a8c36682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.033150 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6f305-c9d8-433f-bcf6-a3b0a8c36682 admin admin] PATH_INFO: `/v3/projects/f0b13519137c44ceb3ad77604ea18913/users/19600de0b1b446c3a366d8a7bb1744f0/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.048149 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6f305-c9d8-433f-bcf6-a3b0a8c36682 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0b13519137c44ceb3ad77604ea18913, user_id=19600de0b1b446c3a366d8a7bb1744f0, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:49.050840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6f305-c9d8-433f-bcf6-a3b0a8c36682 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:49.063115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1276/2534] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:49 2022] PUT /identity/v3/projects/f0b13519137c44ceb3ad77604ea18913/users/19600de0b1b446c3a366d8a7bb1744f0/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:14:49.073315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395e42-acf0-4cf3-ae99-a27222c700e2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.073608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395e42-acf0-4cf3-ae99-a27222c700e2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.074087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395e42-acf0-4cf3-ae99-a27222c700e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.100782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-16395e42-acf0-4cf3-ae99-a27222c700e2 None None] MFA Rules not processed for user `19600de0b1b446c3a366d8a7bb1744f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:49.101983 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-16395e42-acf0-4cf3-ae99-a27222c700e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:49.148074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:49.166927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1259/2535] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:49.171461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6f305-c9d8-433f-bcf6-a3b0a8c36682 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:49.197852 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:49.234679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6f305-c9d8-433f-bcf6-a3b0a8c36682 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:49.239333 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7a290-ce26-4f0c-8553-18bae72ec5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:49.239988 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7a290-ce26-4f0c-8553-18bae72ec5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.240276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7a290-ce26-4f0c-8553-18bae72ec5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.240557 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e7a290-ce26-4f0c-8553-18bae72ec5e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.241248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7a290-ce26-4f0c-8553-18bae72ec5e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:49.243174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e7a290-ce26-4f0c-8553-18bae72ec5e8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:49.268821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1277/2536] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:49 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:49.272848 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16395e42-acf0-4cf3-ae99-a27222c700e2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:49.283300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16395e42-acf0-4cf3-ae99-a27222c700e2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:49.337601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf28c90-a1a6-4f7a-bce6-3ec9c5f36b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:49.338768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf28c90-a1a6-4f7a-bce6-3ec9c5f36b56 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.339053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf28c90-a1a6-4f7a-bce6-3ec9c5f36b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.339329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf28c90-a1a6-4f7a-bce6-3ec9c5f36b56 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.340008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf28c90-a1a6-4f7a-bce6-3ec9c5f36b56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:49.341954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf28c90-a1a6-4f7a-bce6-3ec9c5f36b56 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:49.360883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1260/2537] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:49 2022] POST /identity/v3/users => generated 299 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:49.364822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7a290-ce26-4f0c-8553-18bae72ec5e8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:49.375320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e7a290-ce26-4f0c-8553-18bae72ec5e8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:49.380127 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e70f3a-4dfb-4781-a09c-3cb3fa51e13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:49.380822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e70f3a-4dfb-4781-a09c-3cb3fa51e13b admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.381064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e70f3a-4dfb-4781-a09c-3cb3fa51e13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.381347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e70f3a-4dfb-4781-a09c-3cb3fa51e13b admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.382136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e70f3a-4dfb-4781-a09c-3cb3fa51e13b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:49.383676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e70f3a-4dfb-4781-a09c-3cb3fa51e13b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:49.395827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1278/2538] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:49.399757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf28c90-a1a6-4f7a-bce6-3ec9c5f36b56 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:49.409485 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf28c90-a1a6-4f7a-bce6-3ec9c5f36b56 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:49.414006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f448f13-0c11-4068-af66-f4fe526f5a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:49.414907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f448f13-0c11-4068-af66-f4fe526f5a33 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.415188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f448f13-0c11-4068-af66-f4fe526f5a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.415489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f448f13-0c11-4068-af66-f4fe526f5a33 admin admin] PATH_INFO: `/v3/projects/fdc32e5484274a0fab0ec295c9fd7b60/users/98ff14924aca4181a52aad269e28b2f4/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.429512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f448f13-0c11-4068-af66-f4fe526f5a33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc32e5484274a0fab0ec295c9fd7b60, user_id=98ff14924aca4181a52aad269e28b2f4, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:49.432177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f448f13-0c11-4068-af66-f4fe526f5a33 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:49.445666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1261/2539] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:49 2022] PUT /identity/v3/projects/fdc32e5484274a0fab0ec295c9fd7b60/users/98ff14924aca4181a52aad269e28b2f4/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:49.449526 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e70f3a-4dfb-4781-a09c-3cb3fa51e13b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:49.459790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e70f3a-4dfb-4781-a09c-3cb3fa51e13b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:49.520656 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221ac0ac-d490-4c35-94b6-36bf9267c881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:49.521328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ac0ac-d490-4c35-94b6-36bf9267c881 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.521616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ac0ac-d490-4c35-94b6-36bf9267c881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.521896 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221ac0ac-d490-4c35-94b6-36bf9267c881 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.522652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ac0ac-d490-4c35-94b6-36bf9267c881 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:49.524243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221ac0ac-d490-4c35-94b6-36bf9267c881 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:49.534793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1279/2540] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:49.538815 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f448f13-0c11-4068-af66-f4fe526f5a33 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:49.550674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f448f13-0c11-4068-af66-f4fe526f5a33 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:49.555504 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc4693-f684-4210-a640-341558c94c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:49.556387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc4693-f684-4210-a640-341558c94c15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.556668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc4693-f684-4210-a640-341558c94c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.556941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bc4693-f684-4210-a640-341558c94c15 admin admin] PATH_INFO: `/v3/projects/fdc32e5484274a0fab0ec295c9fd7b60/users/98ff14924aca4181a52aad269e28b2f4/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.558927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc4693-f684-4210-a640-341558c94c15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc32e5484274a0fab0ec295c9fd7b60, user_id=98ff14924aca4181a52aad269e28b2f4, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:49.562063 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bc4693-f684-4210-a640-341558c94c15 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:49.572241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1262/2541] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:49 2022] PUT /identity/v3/projects/fdc32e5484274a0fab0ec295c9fd7b60/users/98ff14924aca4181a52aad269e28b2f4/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:49.584152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1a7ef-2276-4914-8b36-402716534b2a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.584640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1a7ef-2276-4914-8b36-402716534b2a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.584737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1a7ef-2276-4914-8b36-402716534b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.610836 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-58d1a7ef-2276-4914-8b36-402716534b2a None None] MFA Rules not processed for user `98ff14924aca4181a52aad269e28b2f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:49.612044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-58d1a7ef-2276-4914-8b36-402716534b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:49.678968 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1280/2542] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:49.690700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc4693-f684-4210-a640-341558c94c15 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:49.692606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-53bc4693-f684-4210-a640-341558c94c15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:49.749512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bc4693-f684-4210-a640-341558c94c15 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:49.753187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74082aa-34a9-4ba6-9e0b-536371a8932c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:49.753835 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74082aa-34a9-4ba6-9e0b-536371a8932c service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.754420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74082aa-34a9-4ba6-9e0b-536371a8932c service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.754702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74082aa-34a9-4ba6-9e0b-536371a8932c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.763878 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74082aa-34a9-4ba6-9e0b-536371a8932c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:49.765301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74082aa-34a9-4ba6-9e0b-536371a8932c service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:49.774722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1263/2543] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:49.806119 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1a7ef-2276-4914-8b36-402716534b2a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:49.828864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:49.873394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d1a7ef-2276-4914-8b36-402716534b2a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:49.877987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:49.878909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.879477 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.879753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] PATH_INFO: `/v3/users/19600de0b1b446c3a366d8a7bb1744f0` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.881035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] RBAC: Authorizing `identity:delete_user(user_id=19600de0b1b446c3a366d8a7bb1744f0)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:49.882979 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:49.927434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19600de0b1b446c3a366d8a7bb1744f0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:49.928086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19600de0b1b446c3a366d8a7bb1744f0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:49.935455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19600de0b1b446c3a366d8a7bb1744f0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:49.942793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19600de0b1b446c3a366d8a7bb1744f0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:49.943427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19600de0b1b446c3a366d8a7bb1744f0'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:49.960454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1281/2544] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:14:49 2022] DELETE /identity/v3/users/19600de0b1b446c3a366d8a7bb1744f0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:49.964979 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb4485-2427-4678-b76f-de42a4df5459 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:49.965271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb4485-2427-4678-b76f-de42a4df5459 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:49.965569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb4485-2427-4678-b76f-de42a4df5459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:49.983865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-edcb4485-2427-4678-b76f-de42a4df5459 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:49.985050 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-edcb4485-2427-4678-b76f-de42a4df5459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:50.023461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:50.043707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1264/2545] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:50.053680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:50.055165 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:50.110578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ad822-7774-4f77-bfec-0ee3defc507a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:50.114199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057757-aca3-41e6-95aa-5f645808813e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:50.114834 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057757-aca3-41e6-95aa-5f645808813e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:50.115116 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057757-aca3-41e6-95aa-5f645808813e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:50.115392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5057757-aca3-41e6-95aa-5f645808813e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:50.124255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5057757-aca3-41e6-95aa-5f645808813e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:50.125663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5057757-aca3-41e6-95aa-5f645808813e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:50.135185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1282/2546] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:50.390486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb4485-2427-4678-b76f-de42a4df5459 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:50.391922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-edcb4485-2427-4678-b76f-de42a4df5459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:50.404327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb4485-2427-4678-b76f-de42a4df5459 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:50.408884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:50.409753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:50.410059 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:50.410344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] PATH_INFO: `/v3/projects/f0b13519137c44ceb3ad77604ea18913` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:50.411529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0b13519137c44ceb3ad77604ea18913)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:50.413430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:50.459756 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0b13519137c44ceb3ad77604ea18913', 'request_id': 'req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:50.467104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0b13519137c44ceb3ad77604ea18913', 'request_id': 'req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:50.473723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0b13519137c44ceb3ad77604ea18913', 'request_id': 'req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:50.484952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] The token cache is being invalidate because project f0b13519137c44ceb3ad77604ea18913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:14:50.485241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:50.485881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:50.487469 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1265/2547] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:14:50 2022] DELETE /identity/v3/projects/f0b13519137c44ceb3ad77604ea18913 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:50.491578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057757-aca3-41e6-95aa-5f645808813e service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:50.501157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5057757-aca3-41e6-95aa-5f645808813e service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:50.541965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:50.560692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:50.561696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:50.561994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:50.562365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] PATH_INFO: `/v3/users/98ff14924aca4181a52aad269e28b2f4` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:50.563574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98ff14924aca4181a52aad269e28b2f4)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:50.565515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:50.609699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98ff14924aca4181a52aad269e28b2f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:50.610365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98ff14924aca4181a52aad269e28b2f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:50.617072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98ff14924aca4181a52aad269e28b2f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:50.624239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98ff14924aca4181a52aad269e28b2f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:50.624876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-0f851112-5bec-488c-96a5-be104bfa7268 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98ff14924aca4181a52aad269e28b2f4'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:50.642716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1283/2548] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:14:50 2022] DELETE /identity/v3/users/98ff14924aca4181a52aad269e28b2f4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:50.924950 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:50.926680 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:50.941738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe66242-e5df-4bb1-8b75-fa5627c1a5de admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:50.999801 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.000658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.000943 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.001219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] PATH_INFO: `/v3/projects/fdc32e5484274a0fab0ec295c9fd7b60` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.002426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc32e5484274a0fab0ec295c9fd7b60)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.004584 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.050003 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc32e5484274a0fab0ec295c9fd7b60', 'request_id': 'req-b02a74a3-878f-4fd9-b345-0609fd472344'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:51.057094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc32e5484274a0fab0ec295c9fd7b60', 'request_id': 'req-b02a74a3-878f-4fd9-b345-0609fd472344'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:51.063022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc32e5484274a0fab0ec295c9fd7b60', 'request_id': 'req-b02a74a3-878f-4fd9-b345-0609fd472344'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:51.069766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:51.075843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] The token cache is being invalidate because project fdc32e5484274a0fab0ec295c9fd7b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:14:51.076364 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:51.077125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b02a74a3-878f-4fd9-b345-0609fd472344 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:51.079142 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1266/2549] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:14:50 2022] DELETE /identity/v3/projects/fdc32e5484274a0fab0ec295c9fd7b60 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:51.084010 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b02b6d-9b50-4db7-8ad5-1c91ac355bc2 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.084324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b02b6d-9b50-4db7-8ad5-1c91ac355bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.084601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b02b6d-9b50-4db7-8ad5-1c91ac355bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.102141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-42b02b6d-9b50-4db7-8ad5-1c91ac355bc2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:51.103335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-42b02b6d-9b50-4db7-8ad5-1c91ac355bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:51.163771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1284/2550] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:51.172037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4861d0b-e0cd-48e9-9aa2-a8b9f587dcfc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.172328 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4861d0b-e0cd-48e9-9aa2-a8b9f587dcfc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.172609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4861d0b-e0cd-48e9-9aa2-a8b9f587dcfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.190822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f4861d0b-e0cd-48e9-9aa2-a8b9f587dcfc None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:51.192061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f4861d0b-e0cd-48e9-9aa2-a8b9f587dcfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:51.199790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1267/2551] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:51.203932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b02b6d-9b50-4db7-8ad5-1c91ac355bc2 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.214804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b02b6d-9b50-4db7-8ad5-1c91ac355bc2 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.222715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f39ec-7d45-4029-8ad8-6c5090258737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.223632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f39ec-7d45-4029-8ad8-6c5090258737 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.224008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f39ec-7d45-4029-8ad8-6c5090258737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.224197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f39ec-7d45-4029-8ad8-6c5090258737 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.225003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f39ec-7d45-4029-8ad8-6c5090258737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:14:51.225321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f39ec-7d45-4029-8ad8-6c5090258737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.226939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f39ec-7d45-4029-8ad8-6c5090258737 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.239822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1285/2552] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:14:51.244213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4861d0b-e0cd-48e9-9aa2-a8b9f587dcfc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.254435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4861d0b-e0cd-48e9-9aa2-a8b9f587dcfc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.259194 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b328b1b-1026-4ce9-8c45-9a759072117a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.259842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b328b1b-1026-4ce9-8c45-9a759072117a admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.260128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b328b1b-1026-4ce9-8c45-9a759072117a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.260405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b328b1b-1026-4ce9-8c45-9a759072117a admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.261118 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b328b1b-1026-4ce9-8c45-9a759072117a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.263081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b328b1b-1026-4ce9-8c45-9a759072117a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.287586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1268/2553] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:51 2022] POST /identity/v3/projects => generated 387 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:51.292029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f39ec-7d45-4029-8ad8-6c5090258737 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.302414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f39ec-7d45-4029-8ad8-6c5090258737 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.345482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:51.359221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55ce43-9e0c-4a23-987c-e4ea243fdcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.359910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55ce43-9e0c-4a23-987c-e4ea243fdcdc admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.360191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55ce43-9e0c-4a23-987c-e4ea243fdcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.360465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55ce43-9e0c-4a23-987c-e4ea243fdcdc admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.361136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55ce43-9e0c-4a23-987c-e4ea243fdcdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.363083 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c55ce43-9e0c-4a23-987c-e4ea243fdcdc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.384255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1286/2554] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:51 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:51.387817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b328b1b-1026-4ce9-8c45-9a759072117a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.397764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b328b1b-1026-4ce9-8c45-9a759072117a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.402320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4bcfc-e515-46e1-8c10-b665a6249c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.402974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4bcfc-e515-46e1-8c10-b665a6249c52 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.403279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4bcfc-e515-46e1-8c10-b665a6249c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.403557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4bcfc-e515-46e1-8c10-b665a6249c52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.404282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4bcfc-e515-46e1-8c10-b665a6249c52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.405864 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4bcfc-e515-46e1-8c10-b665a6249c52 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.416228 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1269/2555] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:51.420326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55ce43-9e0c-4a23-987c-e4ea243fdcdc admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.430092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c55ce43-9e0c-4a23-987c-e4ea243fdcdc admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.435294 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698a2f1e-b922-4a80-a7a5-0ec68b3c522d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.435940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a2f1e-b922-4a80-a7a5-0ec68b3c522d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.436226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a2f1e-b922-4a80-a7a5-0ec68b3c522d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.436502 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698a2f1e-b922-4a80-a7a5-0ec68b3c522d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.437218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a2f1e-b922-4a80-a7a5-0ec68b3c522d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.438927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698a2f1e-b922-4a80-a7a5-0ec68b3c522d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.450703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1287/2556] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:14:51.454762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4bcfc-e515-46e1-8c10-b665a6249c52 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.464637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4bcfc-e515-46e1-8c10-b665a6249c52 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.469830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a62ff-0341-4b71-b76e-cf1793a8a754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.470701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a62ff-0341-4b71-b76e-cf1793a8a754 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.470980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a62ff-0341-4b71-b76e-cf1793a8a754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.471279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a62ff-0341-4b71-b76e-cf1793a8a754 admin admin] PATH_INFO: `/v3/projects/c087e405dc7c4a3bb4efc155ff543500/users/deb5a26f06744fddbb38896b10240bb9/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.485883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a62ff-0341-4b71-b76e-cf1793a8a754 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c087e405dc7c4a3bb4efc155ff543500, user_id=deb5a26f06744fddbb38896b10240bb9, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.488527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a62ff-0341-4b71-b76e-cf1793a8a754 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.501097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1270/2557] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:51 2022] PUT /identity/v3/projects/c087e405dc7c4a3bb4efc155ff543500/users/deb5a26f06744fddbb38896b10240bb9/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:51.512165 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f481fe-7c46-4826-8a86-a04ef7b0a6f1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.512458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f481fe-7c46-4826-8a86-a04ef7b0a6f1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.512736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f481fe-7c46-4826-8a86-a04ef7b0a6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.538573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-95f481fe-7c46-4826-8a86-a04ef7b0a6f1 None None] MFA Rules not processed for user `deb5a26f06744fddbb38896b10240bb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:51.539822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-95f481fe-7c46-4826-8a86-a04ef7b0a6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:51.609135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1288/2558] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:51.613483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a62ff-0341-4b71-b76e-cf1793a8a754 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.674646 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a62ff-0341-4b71-b76e-cf1793a8a754 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.679271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322a1c7-8e5a-41ed-b880-3585c1ade4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.679923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322a1c7-8e5a-41ed-b880-3585c1ade4b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.680213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322a1c7-8e5a-41ed-b880-3585c1ade4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.680473 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322a1c7-8e5a-41ed-b880-3585c1ade4b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.681160 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322a1c7-8e5a-41ed-b880-3585c1ade4b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.683106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9322a1c7-8e5a-41ed-b880-3585c1ade4b6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.706933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1271/2559] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:51 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:14:51.711197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f481fe-7c46-4826-8a86-a04ef7b0a6f1 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.722876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f481fe-7c46-4826-8a86-a04ef7b0a6f1 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.777984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fcd96-17c6-40e6-90e9-7a979c41d167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.779197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fcd96-17c6-40e6-90e9-7a979c41d167 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.779480 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fcd96-17c6-40e6-90e9-7a979c41d167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.779760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096fcd96-17c6-40e6-90e9-7a979c41d167 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.780446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fcd96-17c6-40e6-90e9-7a979c41d167 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.782375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096fcd96-17c6-40e6-90e9-7a979c41d167 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.802534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1289/2560] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:51 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:51.806377 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322a1c7-8e5a-41ed-b880-3585c1ade4b6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.816473 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9322a1c7-8e5a-41ed-b880-3585c1ade4b6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.821098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb1773-3718-4abb-895a-dfa3506dd29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.821750 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb1773-3718-4abb-895a-dfa3506dd29f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.822032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb1773-3718-4abb-895a-dfa3506dd29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.822338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb1773-3718-4abb-895a-dfa3506dd29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.823071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb1773-3718-4abb-895a-dfa3506dd29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.824655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb1773-3718-4abb-895a-dfa3506dd29f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.835257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1272/2561] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:51.839027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fcd96-17c6-40e6-90e9-7a979c41d167 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.849547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096fcd96-17c6-40e6-90e9-7a979c41d167 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.854925 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c23ea40-2d0b-4d50-8d0d-182117388015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.855139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23ea40-2d0b-4d50-8d0d-182117388015 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.855424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23ea40-2d0b-4d50-8d0d-182117388015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.855704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c23ea40-2d0b-4d50-8d0d-182117388015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.856972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23ea40-2d0b-4d50-8d0d-182117388015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.858605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c23ea40-2d0b-4d50-8d0d-182117388015 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.869587 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1290/2562] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:51.873862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb1773-3718-4abb-895a-dfa3506dd29f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:51.884549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb1773-3718-4abb-895a-dfa3506dd29f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:51.889773 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eebe6e-1477-4ed9-9167-55d50029e4d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:51.890903 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eebe6e-1477-4ed9-9167-55d50029e4d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.891042 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eebe6e-1477-4ed9-9167-55d50029e4d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.891261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eebe6e-1477-4ed9-9167-55d50029e4d2 admin admin] PATH_INFO: `/v3/projects/91a2ce9652df49d6bac635f5e8fafb52/users/8a6deff8d80d48cf9a6838e90b5a8c19/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.906995 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eebe6e-1477-4ed9-9167-55d50029e4d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91a2ce9652df49d6bac635f5e8fafb52, user_id=8a6deff8d80d48cf9a6838e90b5a8c19, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:51.909676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eebe6e-1477-4ed9-9167-55d50029e4d2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:51.922724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1273/2563] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:51 2022] PUT /identity/v3/projects/91a2ce9652df49d6bac635f5e8fafb52/users/8a6deff8d80d48cf9a6838e90b5a8c19/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:14:51.933061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86554cd-2c1a-4504-9ef3-13708a8a9fea None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:51.933352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86554cd-2c1a-4504-9ef3-13708a8a9fea None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:51.933654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86554cd-2c1a-4504-9ef3-13708a8a9fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:51.961050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a86554cd-2c1a-4504-9ef3-13708a8a9fea None None] MFA Rules not processed for user `8a6deff8d80d48cf9a6838e90b5a8c19`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:51.962285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a86554cd-2c1a-4504-9ef3-13708a8a9fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:51.988853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:52.038305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1291/2564] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:51 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:52.043070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eebe6e-1477-4ed9-9167-55d50029e4d2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:52.089595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:52.107396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eebe6e-1477-4ed9-9167-55d50029e4d2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:52.112040 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10829061-307b-433f-9543-63d7e3477751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:52.112702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10829061-307b-433f-9543-63d7e3477751 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:52.112985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10829061-307b-433f-9543-63d7e3477751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:52.113258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10829061-307b-433f-9543-63d7e3477751 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:52.113951 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10829061-307b-433f-9543-63d7e3477751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:52.115976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10829061-307b-433f-9543-63d7e3477751 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:52.145494 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1274/2565] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:14:52 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:14:52.149453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86554cd-2c1a-4504-9ef3-13708a8a9fea None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:52.161962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86554cd-2c1a-4504-9ef3-13708a8a9fea None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:52.220951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d4fa-8288-487a-bf79-0ab7f25b7ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:52.221629 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d4fa-8288-487a-bf79-0ab7f25b7ef3 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:52.221913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d4fa-8288-487a-bf79-0ab7f25b7ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:52.222228 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592d4fa-8288-487a-bf79-0ab7f25b7ef3 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:52.222914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592d4fa-8288-487a-bf79-0ab7f25b7ef3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:52.224817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592d4fa-8288-487a-bf79-0ab7f25b7ef3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:52.248167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1292/2566] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:14:52 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:14:52.249375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10829061-307b-433f-9543-63d7e3477751 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:52.259771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10829061-307b-433f-9543-63d7e3477751 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:52.264975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6571d-854b-4d4d-8078-4f637d57d62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:52.265627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6571d-854b-4d4d-8078-4f637d57d62d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:52.265914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6571d-854b-4d4d-8078-4f637d57d62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:52.266233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad6571d-854b-4d4d-8078-4f637d57d62d admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:52.266952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6571d-854b-4d4d-8078-4f637d57d62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:52.268533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad6571d-854b-4d4d-8078-4f637d57d62d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:52.279507 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1275/2567] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:52.283595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d4fa-8288-487a-bf79-0ab7f25b7ef3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:52.293187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592d4fa-8288-487a-bf79-0ab7f25b7ef3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:52.297671 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f96af-e1b1-4851-9ab2-67221a89e095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:52.298568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f96af-e1b1-4851-9ab2-67221a89e095 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:52.298847 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f96af-e1b1-4851-9ab2-67221a89e095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:52.299117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f96af-e1b1-4851-9ab2-67221a89e095 admin admin] PATH_INFO: `/v3/projects/21fb14cec4a14b75a17657bc51fa3868/users/fe40d9a8ddc046c99b7434a9d6907d06/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:52.313732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f96af-e1b1-4851-9ab2-67221a89e095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21fb14cec4a14b75a17657bc51fa3868, user_id=fe40d9a8ddc046c99b7434a9d6907d06, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:52.316392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f96af-e1b1-4851-9ab2-67221a89e095 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:52.329567 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1293/2568] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:52 2022] PUT /identity/v3/projects/21fb14cec4a14b75a17657bc51fa3868/users/fe40d9a8ddc046c99b7434a9d6907d06/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:52.333434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6571d-854b-4d4d-8078-4f637d57d62d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:52.343550 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad6571d-854b-4d4d-8078-4f637d57d62d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:52.399296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b394338-19cb-4bbf-b739-f4a7998d7794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:52.399961 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394338-19cb-4bbf-b739-f4a7998d7794 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:52.400243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394338-19cb-4bbf-b739-f4a7998d7794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:52.400518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b394338-19cb-4bbf-b739-f4a7998d7794 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:52.401236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b394338-19cb-4bbf-b739-f4a7998d7794 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:52.402884 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b394338-19cb-4bbf-b739-f4a7998d7794 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:52.412600 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1276/2569] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:14:52.416763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f96af-e1b1-4851-9ab2-67221a89e095 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:52.426792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f96af-e1b1-4851-9ab2-67221a89e095 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:52.431304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d19fc7-d744-4c14-ab0c-b5e2eeef1780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:52.432172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d19fc7-d744-4c14-ab0c-b5e2eeef1780 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:52.432451 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d19fc7-d744-4c14-ab0c-b5e2eeef1780 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:52.432728 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d19fc7-d744-4c14-ab0c-b5e2eeef1780 admin admin] PATH_INFO: `/v3/projects/21fb14cec4a14b75a17657bc51fa3868/users/fe40d9a8ddc046c99b7434a9d6907d06/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:52.435174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d19fc7-d744-4c14-ab0c-b5e2eeef1780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21fb14cec4a14b75a17657bc51fa3868, user_id=fe40d9a8ddc046c99b7434a9d6907d06, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:52.437746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d19fc7-d744-4c14-ab0c-b5e2eeef1780 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:52.447683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1294/2570] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:14:52 2022] PUT /identity/v3/projects/21fb14cec4a14b75a17657bc51fa3868/users/fe40d9a8ddc046c99b7434a9d6907d06/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:14:52.458006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2aa6-47c2-43d0-878d-497488db4c86 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:52.458321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2aa6-47c2-43d0-878d-497488db4c86 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:52.458602 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328c2aa6-47c2-43d0-878d-497488db4c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:52.484472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-328c2aa6-47c2-43d0-878d-497488db4c86 None None] MFA Rules not processed for user `fe40d9a8ddc046c99b7434a9d6907d06`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:52.485718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-328c2aa6-47c2-43d0-878d-497488db4c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:52.554822 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1277/2571] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:52 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:14:52.566415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d19fc7-d744-4c14-ab0c-b5e2eeef1780 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:52.568224 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e6d19fc7-d744-4c14-ab0c-b5e2eeef1780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:52.626318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d19fc7-d744-4c14-ab0c-b5e2eeef1780 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:52.630572 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18880d0-7ace-4fce-9747-722ef5fa1e05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:52.631290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18880d0-7ace-4fce-9747-722ef5fa1e05 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:52.631585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18880d0-7ace-4fce-9747-722ef5fa1e05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:52.631959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18880d0-7ace-4fce-9747-722ef5fa1e05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:52.642573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18880d0-7ace-4fce-9747-722ef5fa1e05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:52.644006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18880d0-7ace-4fce-9747-722ef5fa1e05 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:52.654315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1295/2572] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:52 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:52.847524 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c2aa6-47c2-43d0-878d-497488db4c86 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:52.848942 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-328c2aa6-47c2-43d0-878d-497488db4c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:52.894648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:52.910869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328c2aa6-47c2-43d0-878d-497488db4c86 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:52.914818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e84ba-5f88-4e60-a6e6-4698b2db6885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:52.915475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e84ba-5f88-4e60-a6e6-4698b2db6885 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:52.915764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e84ba-5f88-4e60-a6e6-4698b2db6885 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:52.916037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e84ba-5f88-4e60-a6e6-4698b2db6885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:52.917816 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ef6e84ba-5f88-4e60-a6e6-4698b2db6885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:52.997248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e84ba-5f88-4e60-a6e6-4698b2db6885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:52.998734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6e84ba-5f88-4e60-a6e6-4698b2db6885 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:53.009892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1278/2573] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:53.140578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18880d0-7ace-4fce-9747-722ef5fa1e05 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:53.141971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f18880d0-7ace-4fce-9747-722ef5fa1e05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:53.155927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18880d0-7ace-4fce-9747-722ef5fa1e05 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:53.159499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4565b7-d9b3-4b6a-9f8c-83211280b109 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:53.160139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4565b7-d9b3-4b6a-9f8c-83211280b109 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:53.160425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4565b7-d9b3-4b6a-9f8c-83211280b109 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:53.160703 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4565b7-d9b3-4b6a-9f8c-83211280b109 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:53.162053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e84ba-5f88-4e60-a6e6-4698b2db6885 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:53.162716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4d4565b7-d9b3-4b6a-9f8c-83211280b109 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:53.163598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ef6e84ba-5f88-4e60-a6e6-4698b2db6885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:53.176183 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e84ba-5f88-4e60-a6e6-4698b2db6885 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:53.179938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba464967-4135-43f3-833b-3c5fda65ad3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:53.180636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba464967-4135-43f3-833b-3c5fda65ad3e service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:53.180923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba464967-4135-43f3-833b-3c5fda65ad3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:53.181511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba464967-4135-43f3-833b-3c5fda65ad3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:53.181650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:53.228273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4565b7-d9b3-4b6a-9f8c-83211280b109 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:53.229824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4565b7-d9b3-4b6a-9f8c-83211280b109 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:53.239096 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba464967-4135-43f3-833b-3c5fda65ad3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:53.240169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1296/2574] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:53.242517 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba464967-4135-43f3-833b-3c5fda65ad3e service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:53.251875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1279/2575] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:14:56.428357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4565b7-d9b3-4b6a-9f8c-83211280b109 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:56.440115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4565b7-d9b3-4b6a-9f8c-83211280b109 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:56.444692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3316ba4-156c-4f15-b26b-8f7cd935e7a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:56.445399 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3316ba4-156c-4f15-b26b-8f7cd935e7a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:56.445681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3316ba4-156c-4f15-b26b-8f7cd935e7a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:56.445956 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3316ba4-156c-4f15-b26b-8f7cd935e7a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:56.448007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c3316ba4-156c-4f15-b26b-8f7cd935e7a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:56.463783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3316ba4-156c-4f15-b26b-8f7cd935e7a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:56.465245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3316ba4-156c-4f15-b26b-8f7cd935e7a7 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:56.484521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1297/2576] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:57.328319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba464967-4135-43f3-833b-3c5fda65ad3e service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:57.336911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba464967-4135-43f3-833b-3c5fda65ad3e service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:57.341131 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6335f21-d5cf-4ff4-b6f5-58adaa62680d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:57.341764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6335f21-d5cf-4ff4-b6f5-58adaa62680d service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:57.342061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6335f21-d5cf-4ff4-b6f5-58adaa62680d service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:57.342355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6335f21-d5cf-4ff4-b6f5-58adaa62680d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:57.396774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6335f21-d5cf-4ff4-b6f5-58adaa62680d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:57.398252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6335f21-d5cf-4ff4-b6f5-58adaa62680d service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:57.407660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1280/2577] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:14:59.438542 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3316ba4-156c-4f15-b26b-8f7cd935e7a7 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:59.506819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3316ba4-156c-4f15-b26b-8f7cd935e7a7 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:59.511492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:59.512449 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:59.512736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:59.513556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] PATH_INFO: `/v3/users/deb5a26f06744fddbb38896b10240bb9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:59.514859 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deb5a26f06744fddbb38896b10240bb9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:59.516824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:59.568431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deb5a26f06744fddbb38896b10240bb9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:59.568995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deb5a26f06744fddbb38896b10240bb9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:59.576831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deb5a26f06744fddbb38896b10240bb9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:59.584714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deb5a26f06744fddbb38896b10240bb9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:59.585343 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deb5a26f06744fddbb38896b10240bb9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:14:59.603664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1298/2578] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:14:59 2022] DELETE /identity/v3/users/deb5a26f06744fddbb38896b10240bb9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:14:59.608034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bade7b2-ca07-4ad0-8296-d580d79ac5ef None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:59.608559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bade7b2-ca07-4ad0-8296-d580d79ac5ef None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:59.608668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bade7b2-ca07-4ad0-8296-d580d79ac5ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:59.625751 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1bade7b2-ca07-4ad0-8296-d580d79ac5ef None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:14:59.627085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1bade7b2-ca07-4ad0-8296-d580d79ac5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:59.640818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:59.689472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1281/2579] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:14:59.699611 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:14:59.701721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:14:59.725638 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:14:59.770285 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9b9da5-7325-4a25-a0cf-5604aa8a4218 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:14:59.773981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbab5b7-9945-4d94-a53a-e40f89748b4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:14:59.774660 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbab5b7-9945-4d94-a53a-e40f89748b4d service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:14:59.774942 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbab5b7-9945-4d94-a53a-e40f89748b4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:14:59.775215 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbab5b7-9945-4d94-a53a-e40f89748b4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:14:59.784524 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbab5b7-9945-4d94-a53a-e40f89748b4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:14:59.785945 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbab5b7-9945-4d94-a53a-e40f89748b4d service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:14:59.795471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1299/2580] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:14:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:00.084402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bade7b2-ca07-4ad0-8296-d580d79ac5ef None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:00.085919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1bade7b2-ca07-4ad0-8296-d580d79ac5ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:00.097867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bade7b2-ca07-4ad0-8296-d580d79ac5ef None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:00.103254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:00.104449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:00.104804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:00.105086 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] PATH_INFO: `/v3/projects/c087e405dc7c4a3bb4efc155ff543500` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:00.106338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c087e405dc7c4a3bb4efc155ff543500)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:00.109621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:00.157341 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c087e405dc7c4a3bb4efc155ff543500', 'request_id': 'req-bbf67489-5236-498a-8e59-dbe61e489d70'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.164620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c087e405dc7c4a3bb4efc155ff543500', 'request_id': 'req-bbf67489-5236-498a-8e59-dbe61e489d70'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.170691 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c087e405dc7c4a3bb4efc155ff543500', 'request_id': 'req-bbf67489-5236-498a-8e59-dbe61e489d70'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.181252 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] The token cache is being invalidate because project c087e405dc7c4a3bb4efc155ff543500 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:00.181535 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.182182 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.183724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1282/2581] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:00 2022] DELETE /identity/v3/projects/c087e405dc7c4a3bb4efc155ff543500 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:00.188347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbab5b7-9945-4d94-a53a-e40f89748b4d service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:00.198157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbab5b7-9945-4d94-a53a-e40f89748b4d service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:00.253664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:00.254827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:00.255111 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:00.255386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] PATH_INFO: `/v3/users/8a6deff8d80d48cf9a6838e90b5a8c19` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:00.256655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a6deff8d80d48cf9a6838e90b5a8c19)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:00.258729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:00.302422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a6deff8d80d48cf9a6838e90b5a8c19'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.303077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a6deff8d80d48cf9a6838e90b5a8c19'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.310484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a6deff8d80d48cf9a6838e90b5a8c19'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.317431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a6deff8d80d48cf9a6838e90b5a8c19'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.318069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a6deff8d80d48cf9a6838e90b5a8c19'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.335081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1300/2582] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:00 2022] DELETE /identity/v3/users/8a6deff8d80d48cf9a6838e90b5a8c19 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:00.607858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:00.609321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:00.623880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf67489-5236-498a-8e59-dbe61e489d70 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:00.682435 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:00.683313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:00.683595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:00.683868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] PATH_INFO: `/v3/projects/91a2ce9652df49d6bac635f5e8fafb52` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:00.685089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91a2ce9652df49d6bac635f5e8fafb52)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:00.687017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:00.732337 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:00.736615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91a2ce9652df49d6bac635f5e8fafb52', 'request_id': 'req-47bf02bc-949c-413c-a1da-9d927ec93c35'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.745379 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91a2ce9652df49d6bac635f5e8fafb52', 'request_id': 'req-47bf02bc-949c-413c-a1da-9d927ec93c35'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.751421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91a2ce9652df49d6bac635f5e8fafb52', 'request_id': 'req-47bf02bc-949c-413c-a1da-9d927ec93c35'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.763345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] The token cache is being invalidate because project 91a2ce9652df49d6bac635f5e8fafb52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:00.763636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.764409 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.766060 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1283/2583] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:00 2022] DELETE /identity/v3/projects/91a2ce9652df49d6bac635f5e8fafb52 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:00.770147 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:00.771574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:00.786200 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34abf2a6-174e-493b-99ba-679a595afd59 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:00.827603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:00.845992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:00.846898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:00.847177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:00.847453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] PATH_INFO: `/v3/users/fe40d9a8ddc046c99b7434a9d6907d06` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:00.848643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe40d9a8ddc046c99b7434a9d6907d06)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:00.850641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:00.901212 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe40d9a8ddc046c99b7434a9d6907d06'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.901934 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe40d9a8ddc046c99b7434a9d6907d06'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.909625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe40d9a8ddc046c99b7434a9d6907d06'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.917117 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe40d9a8ddc046c99b7434a9d6907d06'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.917733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-553b498f-b8ae-4558-9e34-601de3e484e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe40d9a8ddc046c99b7434a9d6907d06'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:00.935182 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1301/2584] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:00 2022] DELETE /identity/v3/users/fe40d9a8ddc046c99b7434a9d6907d06 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:01.227243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:01.228757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:01.249031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bf02bc-949c-413c-a1da-9d927ec93c35 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:01.307331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:01.308181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.308459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.308730 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] PATH_INFO: `/v3/projects/21fb14cec4a14b75a17657bc51fa3868` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.310432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21fb14cec4a14b75a17657bc51fa3868)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:01.312313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:01.355804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21fb14cec4a14b75a17657bc51fa3868', 'request_id': 'req-ac5f8276-3f6d-4700-b314-fe057d841713'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:01.362005 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21fb14cec4a14b75a17657bc51fa3868', 'request_id': 'req-ac5f8276-3f6d-4700-b314-fe057d841713'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:01.367458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21fb14cec4a14b75a17657bc51fa3868', 'request_id': 'req-ac5f8276-3f6d-4700-b314-fe057d841713'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:01.378566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] The token cache is being invalidate because project 21fb14cec4a14b75a17657bc51fa3868 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:01.378853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:01.379732 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ac5f8276-3f6d-4700-b314-fe057d841713 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:01.381236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1284/2585] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:01 2022] DELETE /identity/v3/projects/21fb14cec4a14b75a17657bc51fa3868 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:01.387281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3e7f0-9a82-4823-86b1-b6fb9720f4dc None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.387580 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3e7f0-9a82-4823-86b1-b6fb9720f4dc None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.388511 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc3e7f0-9a82-4823-86b1-b6fb9720f4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.405599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-dbc3e7f0-9a82-4823-86b1-b6fb9720f4dc None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:01.406848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dbc3e7f0-9a82-4823-86b1-b6fb9720f4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:01.469530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1302/2586] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:01.479080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9894cf-07c9-41b9-9da0-e59822576fa4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.479376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9894cf-07c9-41b9-9da0-e59822576fa4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.479659 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9894cf-07c9-41b9-9da0-e59822576fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.498139 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-cb9894cf-07c9-41b9-9da0-e59822576fa4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:01.499327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cb9894cf-07c9-41b9-9da0-e59822576fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:01.507263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1285/2587] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:01.511282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3e7f0-9a82-4823-86b1-b6fb9720f4dc None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:01.521397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc3e7f0-9a82-4823-86b1-b6fb9720f4dc None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:01.525857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4426c4-d2f2-4bb2-b702-cee2be223149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:01.526545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4426c4-d2f2-4bb2-b702-cee2be223149 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.526827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4426c4-d2f2-4bb2-b702-cee2be223149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.527098 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4426c4-d2f2-4bb2-b702-cee2be223149 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.527806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4426c4-d2f2-4bb2-b702-cee2be223149 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:15:01.528114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4426c4-d2f2-4bb2-b702-cee2be223149 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:01.529905 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4426c4-d2f2-4bb2-b702-cee2be223149 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:01.541146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1303/2588] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:15:01.545280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9894cf-07c9-41b9-9da0-e59822576fa4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:01.554518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9894cf-07c9-41b9-9da0-e59822576fa4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:01.559239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafaf72-4a52-4270-9a2f-681f5ccf3a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:01.560363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafaf72-4a52-4270-9a2f-681f5ccf3a29 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.560648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafaf72-4a52-4270-9a2f-681f5ccf3a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.560924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cafaf72-4a52-4270-9a2f-681f5ccf3a29 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.561613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafaf72-4a52-4270-9a2f-681f5ccf3a29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:01.563645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cafaf72-4a52-4270-9a2f-681f5ccf3a29 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:01.589013 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1286/2589] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:01 2022] POST /identity/v3/projects => generated 409 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:01.592999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4426c4-d2f2-4bb2-b702-cee2be223149 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:01.602892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4426c4-d2f2-4bb2-b702-cee2be223149 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:01.646687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:01.659417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77017821-4806-43db-842b-d1711e8ac4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:01.660055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77017821-4806-43db-842b-d1711e8ac4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.660336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77017821-4806-43db-842b-d1711e8ac4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.660612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77017821-4806-43db-842b-d1711e8ac4f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.661506 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77017821-4806-43db-842b-d1711e8ac4f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:01.663420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77017821-4806-43db-842b-d1711e8ac4f3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:01.682725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1304/2590] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:01 2022] POST /identity/v3/users => generated 305 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:01.687472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafaf72-4a52-4270-9a2f-681f5ccf3a29 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:01.697679 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cafaf72-4a52-4270-9a2f-681f5ccf3a29 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:01.702276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61fa7f-d987-46dd-ac6d-7f01777b112d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:01.702933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61fa7f-d987-46dd-ac6d-7f01777b112d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.703221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61fa7f-d987-46dd-ac6d-7f01777b112d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.703507 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c61fa7f-d987-46dd-ac6d-7f01777b112d admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.704230 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61fa7f-d987-46dd-ac6d-7f01777b112d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:01.705821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c61fa7f-d987-46dd-ac6d-7f01777b112d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:01.717753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1287/2591] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:01.721796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77017821-4806-43db-842b-d1711e8ac4f3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:01.732217 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77017821-4806-43db-842b-d1711e8ac4f3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:01.736783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a9395-ea6e-4e88-92e5-840682f76b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:01.737433 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9395-ea6e-4e88-92e5-840682f76b5d admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.737719 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9395-ea6e-4e88-92e5-840682f76b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.737994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9395-ea6e-4e88-92e5-840682f76b5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.738739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a9395-ea6e-4e88-92e5-840682f76b5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:01.740325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a9395-ea6e-4e88-92e5-840682f76b5d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:01.750801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1305/2592] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:01.755180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61fa7f-d987-46dd-ac6d-7f01777b112d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:01.765066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c61fa7f-d987-46dd-ac6d-7f01777b112d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:01.769629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667962a3-46ed-4675-a4ab-ab28b3f7a845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:01.770537 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667962a3-46ed-4675-a4ab-ab28b3f7a845 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.770830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667962a3-46ed-4675-a4ab-ab28b3f7a845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.771111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667962a3-46ed-4675-a4ab-ab28b3f7a845 admin admin] PATH_INFO: `/v3/projects/97a1c270deb74745a9c82d6c804cb7f6/users/d11296bc15204145bb7c31086fb5401b/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.785484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667962a3-46ed-4675-a4ab-ab28b3f7a845 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a1c270deb74745a9c82d6c804cb7f6, user_id=d11296bc15204145bb7c31086fb5401b, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:01.788141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667962a3-46ed-4675-a4ab-ab28b3f7a845 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:01.796334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:01.802339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1288/2593] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:01 2022] PUT /identity/v3/projects/97a1c270deb74745a9c82d6c804cb7f6/users/d11296bc15204145bb7c31086fb5401b/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:01.814167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77ea7-c355-431a-afe2-b0c46240cb17 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.814677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77ea7-c355-431a-afe2-b0c46240cb17 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.814784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b77ea7-c355-431a-afe2-b0c46240cb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.840280 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-d7b77ea7-c355-431a-afe2-b0c46240cb17 None None] MFA Rules not processed for user `d11296bc15204145bb7c31086fb5401b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:01.841487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d7b77ea7-c355-431a-afe2-b0c46240cb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:01.907762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1306/2594] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:01.912300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667962a3-46ed-4675-a4ab-ab28b3f7a845 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:01.972706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667962a3-46ed-4675-a4ab-ab28b3f7a845 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:01.977378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5f3a7-8cca-4d7a-b66f-e54793441bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:01.978021 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f3a7-8cca-4d7a-b66f-e54793441bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:01.978331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f3a7-8cca-4d7a-b66f-e54793441bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:01.978606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db5f3a7-8cca-4d7a-b66f-e54793441bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:01.979294 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5f3a7-8cca-4d7a-b66f-e54793441bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:01.981161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db5f3a7-8cca-4d7a-b66f-e54793441bb0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:02.006570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1289/2595] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:01 2022] POST /identity/v3/projects => generated 409 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:02.010733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b77ea7-c355-431a-afe2-b0c46240cb17 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:02.021132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b77ea7-c355-431a-afe2-b0c46240cb17 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:02.078118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc39624-2275-4982-ab42-f16ff082808a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:02.078789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc39624-2275-4982-ab42-f16ff082808a admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:02.079074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc39624-2275-4982-ab42-f16ff082808a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:02.079352 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc39624-2275-4982-ab42-f16ff082808a admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:02.080038 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc39624-2275-4982-ab42-f16ff082808a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:02.081948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc39624-2275-4982-ab42-f16ff082808a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:02.101674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1307/2596] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:02 2022] POST /identity/v3/users => generated 311 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:02.105768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5f3a7-8cca-4d7a-b66f-e54793441bb0 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:02.116508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db5f3a7-8cca-4d7a-b66f-e54793441bb0 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:02.121128 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f06ad6-b3d8-42fe-b02b-1eb0266bb709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:02.121782 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f06ad6-b3d8-42fe-b02b-1eb0266bb709 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:02.122115 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f06ad6-b3d8-42fe-b02b-1eb0266bb709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:02.122381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f06ad6-b3d8-42fe-b02b-1eb0266bb709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:02.123106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f06ad6-b3d8-42fe-b02b-1eb0266bb709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:02.124689 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f06ad6-b3d8-42fe-b02b-1eb0266bb709 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:02.135489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1290/2597] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:02.139326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc39624-2275-4982-ab42-f16ff082808a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:02.149341 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc39624-2275-4982-ab42-f16ff082808a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:02.154621 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfc929-7438-4b58-aaaa-e79b8172d14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:02.155497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfc929-7438-4b58-aaaa-e79b8172d14c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:02.155783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfc929-7438-4b58-aaaa-e79b8172d14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:02.156066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dfc929-7438-4b58-aaaa-e79b8172d14c admin admin] PATH_INFO: `/v3/projects/e85a7c725afa4202becc30646046bacc/users/3c7c01842eed4695a738eac02fed1024/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:02.171342 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfc929-7438-4b58-aaaa-e79b8172d14c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85a7c725afa4202becc30646046bacc, user_id=3c7c01842eed4695a738eac02fed1024, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:02.173904 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3dfc929-7438-4b58-aaaa-e79b8172d14c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:02.186211 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1308/2598] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:02 2022] PUT /identity/v3/projects/e85a7c725afa4202becc30646046bacc/users/3c7c01842eed4695a738eac02fed1024/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:02.190008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f06ad6-b3d8-42fe-b02b-1eb0266bb709 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:02.199775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f06ad6-b3d8-42fe-b02b-1eb0266bb709 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:02.255606 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166eedd-d50f-4300-ad67-2a3b9eb77d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:02.256269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166eedd-d50f-4300-ad67-2a3b9eb77d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:02.256558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166eedd-d50f-4300-ad67-2a3b9eb77d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:02.256889 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166eedd-d50f-4300-ad67-2a3b9eb77d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:02.257560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166eedd-d50f-4300-ad67-2a3b9eb77d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:02.259167 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166eedd-d50f-4300-ad67-2a3b9eb77d17 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:02.268967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1291/2599] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:02.273055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfc929-7438-4b58-aaaa-e79b8172d14c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:02.283709 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dfc929-7438-4b58-aaaa-e79b8172d14c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:02.288275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a059d-f000-4672-aa12-4d18fb80cfd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:02.289148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a059d-f000-4672-aa12-4d18fb80cfd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:02.289434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a059d-f000-4672-aa12-4d18fb80cfd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:02.289716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6a059d-f000-4672-aa12-4d18fb80cfd5 admin admin] PATH_INFO: `/v3/projects/e85a7c725afa4202becc30646046bacc/users/3c7c01842eed4695a738eac02fed1024/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:02.291688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a059d-f000-4672-aa12-4d18fb80cfd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e85a7c725afa4202becc30646046bacc, user_id=3c7c01842eed4695a738eac02fed1024, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:02.294952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6a059d-f000-4672-aa12-4d18fb80cfd5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:02.305420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1309/2600] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:02 2022] PUT /identity/v3/projects/e85a7c725afa4202becc30646046bacc/users/3c7c01842eed4695a738eac02fed1024/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:02.316315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801634dd-3f31-4976-bda3-651454991ae4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:02.316629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801634dd-3f31-4976-bda3-651454991ae4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:02.316908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801634dd-3f31-4976-bda3-651454991ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:02.344569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-801634dd-3f31-4976-bda3-651454991ae4 None None] MFA Rules not processed for user `3c7c01842eed4695a738eac02fed1024`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:02.345766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-801634dd-3f31-4976-bda3-651454991ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:02.405739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:02.417217 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1292/2601] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:02 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:02.428405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a059d-f000-4672-aa12-4d18fb80cfd5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:02.430183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4c6a059d-f000-4672-aa12-4d18fb80cfd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:02.441386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:02.494070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6a059d-f000-4672-aa12-4d18fb80cfd5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:02.497647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377359e4-5085-434a-b51f-c1f7023621f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:02.498371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377359e4-5085-434a-b51f-c1f7023621f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:02.498650 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377359e4-5085-434a-b51f-c1f7023621f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:02.498920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377359e4-5085-434a-b51f-c1f7023621f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:02.507951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377359e4-5085-434a-b51f-c1f7023621f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:02.509487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377359e4-5085-434a-b51f-c1f7023621f2 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:02.519122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1310/2602] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:02 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:02.665375 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801634dd-3f31-4976-bda3-651454991ae4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:02.675248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801634dd-3f31-4976-bda3-651454991ae4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:02.678771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa690205-bfda-47f9-a4b9-d6d4470f8b09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:02.679437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690205-bfda-47f9-a4b9-d6d4470f8b09 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:02.679720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690205-bfda-47f9-a4b9-d6d4470f8b09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:02.679997 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa690205-bfda-47f9-a4b9-d6d4470f8b09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:02.735915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa690205-bfda-47f9-a4b9-d6d4470f8b09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:02.737339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa690205-bfda-47f9-a4b9-d6d4470f8b09 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:02.746031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1293/2603] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:12.354993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377359e4-5085-434a-b51f-c1f7023621f2 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:12.414918 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377359e4-5085-434a-b51f-c1f7023621f2 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:12.419435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:12.420281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:12.420560 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:12.420832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] PATH_INFO: `/v3/users/d11296bc15204145bb7c31086fb5401b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:12.421995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d11296bc15204145bb7c31086fb5401b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:12.423906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:12.469531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd11296bc15204145bb7c31086fb5401b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:12.470173 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd11296bc15204145bb7c31086fb5401b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:12.477109 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd11296bc15204145bb7c31086fb5401b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:12.483927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd11296bc15204145bb7c31086fb5401b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:12.484605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd11296bc15204145bb7c31086fb5401b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:12.502291 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1311/2604] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:12 2022] DELETE /identity/v3/users/d11296bc15204145bb7c31086fb5401b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:12.506993 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518576e8-0eda-46b8-8945-7a14472fc3bc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:12.507479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518576e8-0eda-46b8-8945-7a14472fc3bc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:12.507575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518576e8-0eda-46b8-8945-7a14472fc3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:12.524650 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-518576e8-0eda-46b8-8945-7a14472fc3bc None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:12.525843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-518576e8-0eda-46b8-8945-7a14472fc3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:12.583750 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1294/2605] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:12.593436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:12.594872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:12.628471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:12.657863 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b2ed6-7ec8-41e6-a6bb-399438e46a4d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:12.661933 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d51dc-42f3-4f84-9cc1-3d2b902faba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:12.662637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d51dc-42f3-4f84-9cc1-3d2b902faba4 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:12.662951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d51dc-42f3-4f84-9cc1-3d2b902faba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:12.663173 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d51dc-42f3-4f84-9cc1-3d2b902faba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:12.672586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d51dc-42f3-4f84-9cc1-3d2b902faba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:12.673999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d51dc-42f3-4f84-9cc1-3d2b902faba4 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:12.685206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1312/2606] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:12.965866 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518576e8-0eda-46b8-8945-7a14472fc3bc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:12.967923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-518576e8-0eda-46b8-8945-7a14472fc3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:13.000250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518576e8-0eda-46b8-8945-7a14472fc3bc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:13.006445 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:13.007315 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:13.007595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:13.007869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] PATH_INFO: `/v3/projects/97a1c270deb74745a9c82d6c804cb7f6` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:13.009433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a1c270deb74745a9c82d6c804cb7f6)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:13.011378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:13.065080 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a1c270deb74745a9c82d6c804cb7f6', 'request_id': 'req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.074156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a1c270deb74745a9c82d6c804cb7f6', 'request_id': 'req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.080607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a1c270deb74745a9c82d6c804cb7f6', 'request_id': 'req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.092705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] The token cache is being invalidate because project 97a1c270deb74745a9c82d6c804cb7f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:13.092976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.093699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.095624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1295/2607] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:12 2022] DELETE /identity/v3/projects/97a1c270deb74745a9c82d6c804cb7f6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:13.099286 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d51dc-42f3-4f84-9cc1-3d2b902faba4 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:13.110461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d51dc-42f3-4f84-9cc1-3d2b902faba4 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:13.180632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:13.181507 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:13.181787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:13.182088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] PATH_INFO: `/v3/users/3c7c01842eed4695a738eac02fed1024` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:13.183254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c7c01842eed4695a738eac02fed1024)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:13.185878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:13.230730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c7c01842eed4695a738eac02fed1024'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.231324 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c7c01842eed4695a738eac02fed1024'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.238805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c7c01842eed4695a738eac02fed1024'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.247000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c7c01842eed4695a738eac02fed1024'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.247794 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c7c01842eed4695a738eac02fed1024'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.265468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1313/2608] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:13 2022] DELETE /identity/v3/users/3c7c01842eed4695a738eac02fed1024 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:13.541611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:13.543104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:13.556368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d2285-dbc8-4acd-b2c3-c8f471996a13 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:13.576502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:13.622869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:13.623760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:13.624051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:13.624330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] PATH_INFO: `/v3/projects/e85a7c725afa4202becc30646046bacc` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:13.625542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e85a7c725afa4202becc30646046bacc)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:13.627475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:13.674195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e85a7c725afa4202becc30646046bacc', 'request_id': 'req-0d8b1691-8ed7-475e-83b2-983630bf064b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.681255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e85a7c725afa4202becc30646046bacc', 'request_id': 'req-0d8b1691-8ed7-475e-83b2-983630bf064b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.686952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e85a7c725afa4202becc30646046bacc', 'request_id': 'req-0d8b1691-8ed7-475e-83b2-983630bf064b'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.697838 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] The token cache is being invalidate because project e85a7c725afa4202becc30646046bacc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:13.698168 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.698817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:13.700875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1296/2609] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:13 2022] DELETE /identity/v3/projects/e85a7c725afa4202becc30646046bacc => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:13.936830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:13.938349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:13.994614 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad8300-c649-4738-8bdc-7b5abaeffbfc admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:13.999583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:14.048359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d2bb0-2644-4b0c-9297-ddd2246d8320 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:14.048925 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d2bb0-2644-4b0c-9297-ddd2246d8320 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:14.049209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d2bb0-2644-4b0c-9297-ddd2246d8320 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:14.049487 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d2bb0-2644-4b0c-9297-ddd2246d8320 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:14.051268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a02d2bb0-2644-4b0c-9297-ddd2246d8320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:14.112357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d2bb0-2644-4b0c-9297-ddd2246d8320 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:14.113782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d2bb0-2644-4b0c-9297-ddd2246d8320 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:14.123785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1314/2610] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:14.243497 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:14.245107 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:14.258334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8b1691-8ed7-475e-83b2-983630bf064b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:14.262097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4d187-c26b-49c3-8680-185b70eceede service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:14.263416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4d187-c26b-49c3-8680-185b70eceede service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:14.263766 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4d187-c26b-49c3-8680-185b70eceede service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:14.273771 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4d187-c26b-49c3-8680-185b70eceede service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:14.278873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-eeb4d187-c26b-49c3-8680-185b70eceede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:14.291274 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4d187-c26b-49c3-8680-185b70eceede service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:14.292696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4d187-c26b-49c3-8680-185b70eceede service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:14.311887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1297/2611] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:17.854434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d2bb0-2644-4b0c-9297-ddd2246d8320 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:17.855879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a02d2bb0-2644-4b0c-9297-ddd2246d8320 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:17.898824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4d187-c26b-49c3-8680-185b70eceede service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:17.900502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-eeb4d187-c26b-49c3-8680-185b70eceede service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:17.916135 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4d187-c26b-49c3-8680-185b70eceede service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:17.926299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:17.928368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:17.928738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:17.928940 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] PATH_INFO: `/v3/users/430d2efc691e4aae9f96409b081c7e96` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:17.951612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] RBAC: Authorizing `identity:delete_user(user_id=430d2efc691e4aae9f96409b081c7e96)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:17.953809 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:17.962971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d2bb0-2644-4b0c-9297-ddd2246d8320 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:17.969626 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c4e96-5c81-40de-83d3-62b61b563eb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:17.970322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c4e96-5c81-40de-83d3-62b61b563eb2 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:17.970607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c4e96-5c81-40de-83d3-62b61b563eb2 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:17.970881 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c4e96-5c81-40de-83d3-62b61b563eb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:17.973430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4b2c4e96-5c81-40de-83d3-62b61b563eb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:17.988072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c4e96-5c81-40de-83d3-62b61b563eb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:17.990105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c4e96-5c81-40de-83d3-62b61b563eb2 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.002395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1315/2612] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:18.014098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '430d2efc691e4aae9f96409b081c7e96'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.014098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '430d2efc691e4aae9f96409b081c7e96'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.014098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '430d2efc691e4aae9f96409b081c7e96'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.031415 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '430d2efc691e4aae9f96409b081c7e96'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.040545 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '430d2efc691e4aae9f96409b081c7e96'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.046004 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:18.057240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1298/2613] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:17 2022] DELETE /identity/v3/users/430d2efc691e4aae9f96409b081c7e96 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:18.145229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c4e96-5c81-40de-83d3-62b61b563eb2 service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.146681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4b2c4e96-5c81-40de-83d3-62b61b563eb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:18.162444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c4e96-5c81-40de-83d3-62b61b563eb2 service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.207754 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=100 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:18.220610 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.221495 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.221781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.222087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] PATH_INFO: `/v3/users/3b7a90ca5e374fc882b644e34e0a3015` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.237460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b7a90ca5e374fc882b644e34e0a3015)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.239682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.259242 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.260704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:18.275778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d4966-f04f-4206-82bc-4bf45fdbc10a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.282245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.283180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.283496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.283785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] PATH_INFO: `/v3/projects/5dd1a7a651fa4d27a1b6488ca13effca` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.298113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dd1a7a651fa4d27a1b6488ca13effca)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.300285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.300936 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b7a90ca5e374fc882b644e34e0a3015'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.301701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b7a90ca5e374fc882b644e34e0a3015'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.308534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b7a90ca5e374fc882b644e34e0a3015'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.316760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b7a90ca5e374fc882b644e34e0a3015'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.317641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-43500655-de2a-4f45-8621-90f5ddd3f72a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b7a90ca5e374fc882b644e34e0a3015'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.337714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1316/2614] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:18 2022] DELETE /identity/v3/users/3b7a90ca5e374fc882b644e34e0a3015 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:18.350258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dd1a7a651fa4d27a1b6488ca13effca', 'request_id': 'req-82e8f14f-eee9-4176-b150-76d2f77079be'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.357798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dd1a7a651fa4d27a1b6488ca13effca', 'request_id': 'req-82e8f14f-eee9-4176-b150-76d2f77079be'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.363816 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dd1a7a651fa4d27a1b6488ca13effca', 'request_id': 'req-82e8f14f-eee9-4176-b150-76d2f77079be'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.376955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] The token cache is being invalidate because project 5dd1a7a651fa4d27a1b6488ca13effca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:18.377363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.378133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-82e8f14f-eee9-4176-b150-76d2f77079be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.380200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1299/2615] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:18 2022] DELETE /identity/v3/projects/5dd1a7a651fa4d27a1b6488ca13effca => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:18.384824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dbe73-10cb-402b-b08b-af3c25ca2be6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.385122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dbe73-10cb-402b-b08b-af3c25ca2be6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.385402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034dbe73-10cb-402b-b08b-af3c25ca2be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.404741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-034dbe73-10cb-402b-b08b-af3c25ca2be6 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:18.405888 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-034dbe73-10cb-402b-b08b-af3c25ca2be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:18.466114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1317/2616] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:18.473999 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4bc6-d13a-4480-981f-4dbbcfa774ff None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.474325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4bc6-d13a-4480-981f-4dbbcfa774ff None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.474975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90da4bc6-d13a-4480-981f-4dbbcfa774ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.492258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-90da4bc6-d13a-4480-981f-4dbbcfa774ff None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:18.493463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-90da4bc6-d13a-4480-981f-4dbbcfa774ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:18.502339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1300/2617] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:18.506906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dbe73-10cb-402b-b08b-af3c25ca2be6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.517559 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034dbe73-10cb-402b-b08b-af3c25ca2be6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.523049 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da4bc6-d13a-4480-981f-4dbbcfa774ff None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.524655 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-90da4bc6-d13a-4480-981f-4dbbcfa774ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:18.527986 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a92ed-3848-4e65-b3cd-6c24cfb1196f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.528697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a92ed-3848-4e65-b3cd-6c24cfb1196f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.528981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a92ed-3848-4e65-b3cd-6c24cfb1196f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.529255 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a92ed-3848-4e65-b3cd-6c24cfb1196f admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.530238 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a92ed-3848-4e65-b3cd-6c24cfb1196f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:15:18.530377 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a92ed-3848-4e65-b3cd-6c24cfb1196f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.531963 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a92ed-3848-4e65-b3cd-6c24cfb1196f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.540956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90da4bc6-d13a-4480-981f-4dbbcfa774ff None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.544563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1318/2618] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:15:18.546137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.546991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.547300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.547578 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] PATH_INFO: `/v3/projects/1850f9f425294a82add8a09690dc1df0` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.548737 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a92ed-3848-4e65-b3cd-6c24cfb1196f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.559967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a92ed-3848-4e65-b3cd-6c24cfb1196f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.561428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1850f9f425294a82add8a09690dc1df0)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.563489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.564841 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f76fe-58b8-439b-8799-61b6abaa45cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.565597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f76fe-58b8-439b-8799-61b6abaa45cc admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.565961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f76fe-58b8-439b-8799-61b6abaa45cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.566277 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f76fe-58b8-439b-8799-61b6abaa45cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.566965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f76fe-58b8-439b-8799-61b6abaa45cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.568839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f76fe-58b8-439b-8799-61b6abaa45cc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.596337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1319/2619] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:18 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:18.600514 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f76fe-58b8-439b-8799-61b6abaa45cc admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.620302 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1850f9f425294a82add8a09690dc1df0', 'request_id': 'req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.628234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:18.629344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1850f9f425294a82add8a09690dc1df0', 'request_id': 'req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.636546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1850f9f425294a82add8a09690dc1df0', 'request_id': 'req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.649042 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] The token cache is being invalidate because project 1850f9f425294a82add8a09690dc1df0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:18.649432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.650136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-acacb83e-6dd8-45b3-ba2e-3e52aab423b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:18.651727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1301/2620] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:18 2022] DELETE /identity/v3/projects/1850f9f425294a82add8a09690dc1df0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:18.656475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be5233-4fbf-42fb-9034-16e4b632f137 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.656763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be5233-4fbf-42fb-9034-16e4b632f137 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.657053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be5233-4fbf-42fb-9034-16e4b632f137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.668245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f76fe-58b8-439b-8799-61b6abaa45cc admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.675571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-27be5233-4fbf-42fb-9034-16e4b632f137 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:18.677413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-27be5233-4fbf-42fb-9034-16e4b632f137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:18.717685 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:18.731628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9b501-0c5a-4ac0-93bd-10ad6016167f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.732347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9b501-0c5a-4ac0-93bd-10ad6016167f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.732648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9b501-0c5a-4ac0-93bd-10ad6016167f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.732971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b9b501-0c5a-4ac0-93bd-10ad6016167f admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.733676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9b501-0c5a-4ac0-93bd-10ad6016167f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.735748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b9b501-0c5a-4ac0-93bd-10ad6016167f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.753376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1302/2621] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:15:18.761648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdeb14-2865-4c09-9140-0b0c9efffdbc None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.762026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdeb14-2865-4c09-9140-0b0c9efffdbc None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.762430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebdeb14-2865-4c09-9140-0b0c9efffdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.766270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1320/2622] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:18 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:18.767259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9b501-0c5a-4ac0-93bd-10ad6016167f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.768924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-55b9b501-0c5a-4ac0-93bd-10ad6016167f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:18.782073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b9b501-0c5a-4ac0-93bd-10ad6016167f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.783740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-eebdeb14-2865-4c09-9140-0b0c9efffdbc None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:18.784965 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-eebdeb14-2865-4c09-9140-0b0c9efffdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:18.790430 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86616b-f48a-4900-bc12-f0afbc3a9a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.791170 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86616b-f48a-4900-bc12-f0afbc3a9a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.791525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86616b-f48a-4900-bc12-f0afbc3a9a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.791884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d86616b-f48a-4900-bc12-f0afbc3a9a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.792711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86616b-f48a-4900-bc12-f0afbc3a9a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.793213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1303/2623] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:18.795466 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d86616b-f48a-4900-bc12-f0afbc3a9a26 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.797580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdeb14-2865-4c09-9140-0b0c9efffdbc None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.807642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebdeb14-2865-4c09-9140-0b0c9efffdbc None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.811276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1321/2624] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:15:18.815015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86616b-f48a-4900-bc12-f0afbc3a9a26 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.816046 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04a7bb-28b3-4807-bf1b-21c9fa5408d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.816708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04a7bb-28b3-4807-bf1b-21c9fa5408d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.816989 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04a7bb-28b3-4807-bf1b-21c9fa5408d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.817262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04a7bb-28b3-4807-bf1b-21c9fa5408d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.818014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04a7bb-28b3-4807-bf1b-21c9fa5408d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:15:18.818360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04a7bb-28b3-4807-bf1b-21c9fa5408d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.819910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04a7bb-28b3-4807-bf1b-21c9fa5408d6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.825930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d86616b-f48a-4900-bc12-f0afbc3a9a26 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.830750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeba18d-64e0-45e8-a48f-c4660f5df87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.831403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeba18d-64e0-45e8-a48f-c4660f5df87e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.831688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeba18d-64e0-45e8-a48f-c4660f5df87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.831964 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eeba18d-64e0-45e8-a48f-c4660f5df87e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.835916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1304/2625] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:15:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:15:18.836055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeba18d-64e0-45e8-a48f-c4660f5df87e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.836055 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eeba18d-64e0-45e8-a48f-c4660f5df87e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.836840 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04a7bb-28b3-4807-bf1b-21c9fa5408d6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.849223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1322/2626] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:18.849493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04a7bb-28b3-4807-bf1b-21c9fa5408d6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.850213 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeba18d-64e0-45e8-a48f-c4660f5df87e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.853146 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e42c6a-e492-4be1-b406-0ea34059b905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.853986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e42c6a-e492-4be1-b406-0ea34059b905 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.854595 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e42c6a-e492-4be1-b406-0ea34059b905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.854991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e42c6a-e492-4be1-b406-0ea34059b905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.855694 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e42c6a-e492-4be1-b406-0ea34059b905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.858226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e42c6a-e492-4be1-b406-0ea34059b905 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.860750 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eeba18d-64e0-45e8-a48f-c4660f5df87e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:18.865713 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10b4ed4-79bd-4042-bd30-52a5bfedf79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:18.866627 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b4ed4-79bd-4042-bd30-52a5bfedf79b admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.866914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b4ed4-79bd-4042-bd30-52a5bfedf79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.867197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10b4ed4-79bd-4042-bd30-52a5bfedf79b admin admin] PATH_INFO: `/v3/projects/1e471a5e4aa948af9466c062b550d035/users/3a27b452cbee46cdb7a37752f554527d/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.883820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b4ed4-79bd-4042-bd30-52a5bfedf79b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e471a5e4aa948af9466c062b550d035, user_id=3a27b452cbee46cdb7a37752f554527d, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:18.886493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10b4ed4-79bd-4042-bd30-52a5bfedf79b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:18.889954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1305/2627] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:18 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:18.891186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e42c6a-e492-4be1-b406-0ea34059b905 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:18.899685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1323/2628] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:18 2022] PUT /identity/v3/projects/1e471a5e4aa948af9466c062b550d035/users/3a27b452cbee46cdb7a37752f554527d/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:18.909925 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1a996-8152-4ce3-95bd-77acf3ce659f None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:18.910838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1a996-8152-4ce3-95bd-77acf3ce659f None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:18.911124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc1a996-8152-4ce3-95bd-77acf3ce659f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:18.937356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9fc1a996-8152-4ce3-95bd-77acf3ce659f None None] MFA Rules not processed for user `3a27b452cbee46cdb7a37752f554527d`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:18.938615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9fc1a996-8152-4ce3-95bd-77acf3ce659f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:18.954283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e42c6a-e492-4be1-b406-0ea34059b905 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.001913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1324/2629] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:15:19.006757 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1a996-8152-4ce3-95bd-77acf3ce659f None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.010471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:19.011186 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f9182-316a-43ba-b15d-8a0c0ed36b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.012014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f9182-316a-43ba-b15d-8a0c0ed36b2c admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.012199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f9182-316a-43ba-b15d-8a0c0ed36b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.013325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f9182-316a-43ba-b15d-8a0c0ed36b2c admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.014422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f9182-316a-43ba-b15d-8a0c0ed36b2c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.016397 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f9182-316a-43ba-b15d-8a0c0ed36b2c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.042074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1306/2630] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:18 2022] POST /identity/v3/users => generated 279 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:19.044285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f9182-316a-43ba-b15d-8a0c0ed36b2c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.055326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f9182-316a-43ba-b15d-8a0c0ed36b2c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.060078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05770bf2-0770-47a0-84d2-63aabb9a2033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.060737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05770bf2-0770-47a0-84d2-63aabb9a2033 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.061023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05770bf2-0770-47a0-84d2-63aabb9a2033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.061302 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05770bf2-0770-47a0-84d2-63aabb9a2033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.062073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05770bf2-0770-47a0-84d2-63aabb9a2033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.063675 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05770bf2-0770-47a0-84d2-63aabb9a2033 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.077220 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1307/2631] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:15:19.077417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc1a996-8152-4ce3-95bd-77acf3ce659f None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.078393 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05770bf2-0770-47a0-84d2-63aabb9a2033 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.081226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e5fb7-4279-4099-a5eb-d9bb7c0840fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.081877 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e5fb7-4279-4099-a5eb-d9bb7c0840fa admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.082193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e5fb7-4279-4099-a5eb-d9bb7c0840fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.082466 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e5fb7-4279-4099-a5eb-d9bb7c0840fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.083147 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e5fb7-4279-4099-a5eb-d9bb7c0840fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.085044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e5fb7-4279-4099-a5eb-d9bb7c0840fa admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.088036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05770bf2-0770-47a0-84d2-63aabb9a2033 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.093271 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecbfd8-c9bb-4e5d-8fcb-845bec92614a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.093914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecbfd8-c9bb-4e5d-8fcb-845bec92614a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.094239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecbfd8-c9bb-4e5d-8fcb-845bec92614a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.094522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ecbfd8-c9bb-4e5d-8fcb-845bec92614a admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.095233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecbfd8-c9bb-4e5d-8fcb-845bec92614a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.096793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ecbfd8-c9bb-4e5d-8fcb-845bec92614a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.108311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1308/2632] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:19.112293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecbfd8-c9bb-4e5d-8fcb-845bec92614a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.114306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1325/2633] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:19 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:19.117919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e5fb7-4279-4099-a5eb-d9bb7c0840fa admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.122555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ecbfd8-c9bb-4e5d-8fcb-845bec92614a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.127749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:19.180382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e5fb7-4279-4099-a5eb-d9bb7c0840fa admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.182580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f1c2e0-041c-474b-8d06-039620af3d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.183468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1c2e0-041c-474b-8d06-039620af3d0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.183760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1c2e0-041c-474b-8d06-039620af3d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.184027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f1c2e0-041c-474b-8d06-039620af3d0d admin admin] PATH_INFO: `/v3/projects/12792c1e562b48e5bc59f61b8c1c10f8/users/e9fad83651d64ffb9d39d71c4baec116/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.185997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc92bde-9503-4672-b7db-8ad4fe8a65ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.186690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc92bde-9503-4672-b7db-8ad4fe8a65ba admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.186970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc92bde-9503-4672-b7db-8ad4fe8a65ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.187268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc92bde-9503-4672-b7db-8ad4fe8a65ba admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.187952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc92bde-9503-4672-b7db-8ad4fe8a65ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.189854 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc92bde-9503-4672-b7db-8ad4fe8a65ba admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.199282 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1c2e0-041c-474b-8d06-039620af3d0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12792c1e562b48e5bc59f61b8c1c10f8, user_id=e9fad83651d64ffb9d39d71c4baec116, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.202632 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f1c2e0-041c-474b-8d06-039620af3d0d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.210996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1326/2634] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:19 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:19.214797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc92bde-9503-4672-b7db-8ad4fe8a65ba admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.215727 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1309/2635] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:19 2022] PUT /identity/v3/projects/12792c1e562b48e5bc59f61b8c1c10f8/users/e9fad83651d64ffb9d39d71c4baec116/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:19.224581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc92bde-9503-4672-b7db-8ad4fe8a65ba admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.225373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd84f-cfae-42c3-96f9-88f4119a879b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.225706 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd84f-cfae-42c3-96f9-88f4119a879b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.225952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcd84f-cfae-42c3-96f9-88f4119a879b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.251864 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d0dcd84f-cfae-42c3-96f9-88f4119a879b None None] MFA Rules not processed for user `e9fad83651d64ffb9d39d71c4baec116`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:19.253187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d0dcd84f-cfae-42c3-96f9-88f4119a879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:19.285868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6554d11-8ef7-4c8a-81fb-7c93551fab71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.286735 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6554d11-8ef7-4c8a-81fb-7c93551fab71 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.287013 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6554d11-8ef7-4c8a-81fb-7c93551fab71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.287290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6554d11-8ef7-4c8a-81fb-7c93551fab71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.288011 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6554d11-8ef7-4c8a-81fb-7c93551fab71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.289589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6554d11-8ef7-4c8a-81fb-7c93551fab71 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.300363 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1327/2636] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:15:19.304347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6554d11-8ef7-4c8a-81fb-7c93551fab71 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.314279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6554d11-8ef7-4c8a-81fb-7c93551fab71 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.318995 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54b80d-619c-4b17-a8eb-b7a264090f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.319918 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54b80d-619c-4b17-a8eb-b7a264090f61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.320204 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54b80d-619c-4b17-a8eb-b7a264090f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.320478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b54b80d-619c-4b17-a8eb-b7a264090f61 admin admin] PATH_INFO: `/v3/projects/d3c1952100194b52b2dab5bc43e9bfb6/users/9b9b54f1a5cc4d48a6369a50a611fdb9/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.320690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1310/2637] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:19 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:15:19.324906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd84f-cfae-42c3-96f9-88f4119a879b None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.334428 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:19.335698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcd84f-cfae-42c3-96f9-88f4119a879b None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.337930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54b80d-619c-4b17-a8eb-b7a264090f61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c1952100194b52b2dab5bc43e9bfb6, user_id=9b9b54f1a5cc4d48a6369a50a611fdb9, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.340763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaa591-f902-43ab-86c0-7a50e2d17f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.341233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b54b80d-619c-4b17-a8eb-b7a264090f61 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.341410 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaa591-f902-43ab-86c0-7a50e2d17f1e admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.341698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaa591-f902-43ab-86c0-7a50e2d17f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.342007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbaa591-f902-43ab-86c0-7a50e2d17f1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.342875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbaa591-f902-43ab-86c0-7a50e2d17f1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.344904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbaa591-f902-43ab-86c0-7a50e2d17f1e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.357093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1328/2638] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:19 2022] PUT /identity/v3/projects/d3c1952100194b52b2dab5bc43e9bfb6/users/9b9b54f1a5cc4d48a6369a50a611fdb9/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:19.360724 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54b80d-619c-4b17-a8eb-b7a264090f61 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.372881 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1311/2639] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:19 2022] POST /identity/v3/projects => generated 355 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:19.376823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaa591-f902-43ab-86c0-7a50e2d17f1e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.422633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b54b80d-619c-4b17-a8eb-b7a264090f61 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.437096 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbaa591-f902-43ab-86c0-7a50e2d17f1e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.441823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95350cdf-8439-4f8e-a47d-02c30e6e9177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.442504 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95350cdf-8439-4f8e-a47d-02c30e6e9177 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.442784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95350cdf-8439-4f8e-a47d-02c30e6e9177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.443058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95350cdf-8439-4f8e-a47d-02c30e6e9177 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.443734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95350cdf-8439-4f8e-a47d-02c30e6e9177 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.445623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95350cdf-8439-4f8e-a47d-02c30e6e9177 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.452778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:19.470148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1312/2640] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:19 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:19.471136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95350cdf-8439-4f8e-a47d-02c30e6e9177 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.479533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fbbe0-f626-40be-9f9b-7bf4823f9ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.480375 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fbbe0-f626-40be-9f9b-7bf4823f9ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.480815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fbbe0-f626-40be-9f9b-7bf4823f9ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.481136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530fbbe0-f626-40be-9f9b-7bf4823f9ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.481260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95350cdf-8439-4f8e-a47d-02c30e6e9177 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.482191 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fbbe0-f626-40be-9f9b-7bf4823f9ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.483786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530fbbe0-f626-40be-9f9b-7bf4823f9ea4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.486515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3833f-01a9-4db7-be48-7cac058a6679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.487163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3833f-01a9-4db7-be48-7cac058a6679 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.487452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3833f-01a9-4db7-be48-7cac058a6679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.487732 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3833f-01a9-4db7-be48-7cac058a6679 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.488455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3833f-01a9-4db7-be48-7cac058a6679 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.490020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3833f-01a9-4db7-be48-7cac058a6679 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.494094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1329/2641] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:15:19.498493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fbbe0-f626-40be-9f9b-7bf4823f9ea4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.504376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1313/2642] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:19.505280 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3833f-01a9-4db7-be48-7cac058a6679 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.509688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530fbbe0-f626-40be-9f9b-7bf4823f9ea4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.515243 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3833f-01a9-4db7-be48-7cac058a6679 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.515437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8910ff3c-ca72-4e55-b0d4-610a148bf096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.516447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910ff3c-ca72-4e55-b0d4-610a148bf096 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.516728 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910ff3c-ca72-4e55-b0d4-610a148bf096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.516998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8910ff3c-ca72-4e55-b0d4-610a148bf096 admin admin] PATH_INFO: `/v3/projects/d3c1952100194b52b2dab5bc43e9bfb6/users/9b9b54f1a5cc4d48a6369a50a611fdb9/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.518971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910ff3c-ca72-4e55-b0d4-610a148bf096 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c1952100194b52b2dab5bc43e9bfb6, user_id=9b9b54f1a5cc4d48a6369a50a611fdb9, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.519963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6294b1-05f8-48ca-9676-2546b7b06745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.520813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6294b1-05f8-48ca-9676-2546b7b06745 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.521097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6294b1-05f8-48ca-9676-2546b7b06745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.521372 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6294b1-05f8-48ca-9676-2546b7b06745 admin admin] PATH_INFO: `/v3/projects/85933743b9fa4d28a9f5c876bd958242/users/c4231725cd854a49b45fa339f8dc81a4/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.521570 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8910ff3c-ca72-4e55-b0d4-610a148bf096 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.532432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1330/2643] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:19 2022] PUT /identity/v3/projects/d3c1952100194b52b2dab5bc43e9bfb6/users/9b9b54f1a5cc4d48a6369a50a611fdb9/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:19.536333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6294b1-05f8-48ca-9676-2546b7b06745 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85933743b9fa4d28a9f5c876bd958242, user_id=c4231725cd854a49b45fa339f8dc81a4, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.538981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6294b1-05f8-48ca-9676-2546b7b06745 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.543520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7516-94cb-4866-9bf8-005b72fff79b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.543838 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7516-94cb-4866-9bf8-005b72fff79b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.544391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bf7516-94cb-4866-9bf8-005b72fff79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.553169 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1314/2644] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:19 2022] PUT /identity/v3/projects/85933743b9fa4d28a9f5c876bd958242/users/c4231725cd854a49b45fa339f8dc81a4/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:19.556960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6294b1-05f8-48ca-9676-2546b7b06745 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.572661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-41bf7516-94cb-4866-9bf8-005b72fff79b None None] MFA Rules not processed for user `9b9b54f1a5cc4d48a6369a50a611fdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:19.574220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-41bf7516-94cb-4866-9bf8-005b72fff79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:19.618162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6294b1-05f8-48ca-9676-2546b7b06745 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.623581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed0a21-626a-4ff0-82e8-e3a6d2b80e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.624266 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed0a21-626a-4ff0-82e8-e3a6d2b80e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.624548 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed0a21-626a-4ff0-82e8-e3a6d2b80e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.624824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ed0a21-626a-4ff0-82e8-e3a6d2b80e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.625584 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed0a21-626a-4ff0-82e8-e3a6d2b80e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.627197 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ed0a21-626a-4ff0-82e8-e3a6d2b80e54 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.637437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1315/2645] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:15:19.641869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed0a21-626a-4ff0-82e8-e3a6d2b80e54 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.644409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1331/2646] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:19.651707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ed0a21-626a-4ff0-82e8-e3a6d2b80e54 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.654772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf7516-94cb-4866-9bf8-005b72fff79b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.656115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-41bf7516-94cb-4866-9bf8-005b72fff79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:19.656688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c20646f-4bbc-42ca-80d0-5d68115d32ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.657596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20646f-4bbc-42ca-80d0-5d68115d32ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.657887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20646f-4bbc-42ca-80d0-5d68115d32ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.658205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c20646f-4bbc-42ca-80d0-5d68115d32ee admin admin] PATH_INFO: `/v3/projects/85933743b9fa4d28a9f5c876bd958242/users/c4231725cd854a49b45fa339f8dc81a4/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.660233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20646f-4bbc-42ca-80d0-5d68115d32ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=85933743b9fa4d28a9f5c876bd958242, user_id=c4231725cd854a49b45fa339f8dc81a4, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.662406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:19.662954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c20646f-4bbc-42ca-80d0-5d68115d32ee admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.674843 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1316/2647] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:19 2022] PUT /identity/v3/projects/85933743b9fa4d28a9f5c876bd958242/users/c4231725cd854a49b45fa339f8dc81a4/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:19.686122 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db883b6-eccb-4912-a4db-5aa275887e3c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.686421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db883b6-eccb-4912-a4db-5aa275887e3c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.686701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db883b6-eccb-4912-a4db-5aa275887e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.712910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-3db883b6-eccb-4912-a4db-5aa275887e3c None None] MFA Rules not processed for user `c4231725cd854a49b45fa339f8dc81a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:19.714104 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3db883b6-eccb-4912-a4db-5aa275887e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:19.721724 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bf7516-94cb-4866-9bf8-005b72fff79b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.772619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5087c67-00c7-4f05-af4c-76be788bac79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.773267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5087c67-00c7-4f05-af4c-76be788bac79 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.773987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5087c67-00c7-4f05-af4c-76be788bac79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.773987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5087c67-00c7-4f05-af4c-76be788bac79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.782346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1317/2648] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:19 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:15:19.793400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db883b6-eccb-4912-a4db-5aa275887e3c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.795270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-3db883b6-eccb-4912-a4db-5aa275887e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:19.806683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db883b6-eccb-4912-a4db-5aa275887e3c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.810851 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5df79-238b-46f0-9731-b44e1a1cd3ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.811484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5df79-238b-46f0-9731-b44e1a1cd3ea service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.811768 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5df79-238b-46f0-9731-b44e1a1cd3ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.812047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b5df79-238b-46f0-9731-b44e1a1cd3ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.821451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5df79-238b-46f0-9731-b44e1a1cd3ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.822887 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b5df79-238b-46f0-9731-b44e1a1cd3ea service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.831362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5087c67-00c7-4f05-af4c-76be788bac79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.832305 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1318/2649] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:19 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:15:19.832850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5087c67-00c7-4f05-af4c-76be788bac79 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.842388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1332/2650] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:19 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:15:19.905512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5df79-238b-46f0-9731-b44e1a1cd3ea service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:19.915058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b5df79-238b-46f0-9731-b44e1a1cd3ea service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:19.919181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13038a26-b49b-4a43-be52-0a4caf369831 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:19.920006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13038a26-b49b-4a43-be52-0a4caf369831 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:19.920296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13038a26-b49b-4a43-be52-0a4caf369831 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:19.920580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13038a26-b49b-4a43-be52-0a4caf369831 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:19.925962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:19.980932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13038a26-b49b-4a43-be52-0a4caf369831 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:19.982921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13038a26-b49b-4a43-be52-0a4caf369831 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:19.992113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1319/2651] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:19 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:25.522600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5087c67-00c7-4f05-af4c-76be788bac79 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:25.531553 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5087c67-00c7-4f05-af4c-76be788bac79 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:25.535534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb61a1-cef1-49fb-8b18-1f353390f7f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:25.536156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb61a1-cef1-49fb-8b18-1f353390f7f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:25.536437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb61a1-cef1-49fb-8b18-1f353390f7f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:25.536710 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb61a1-cef1-49fb-8b18-1f353390f7f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:25.596573 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb61a1-cef1-49fb-8b18-1f353390f7f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:25.597999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb61a1-cef1-49fb-8b18-1f353390f7f8 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:25.607415 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1333/2652] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:29.988285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=24 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:35.603131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:45.308508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13038a26-b49b-4a43-be52-0a4caf369831 service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:45.374945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13038a26-b49b-4a43-be52-0a4caf369831 service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:45.379582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:45.380457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:45.380742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:45.381018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] PATH_INFO: `/v3/users/e9fad83651d64ffb9d39d71c4baec116` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:45.382235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9fad83651d64ffb9d39d71c4baec116)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:45.384140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:45.432604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9fad83651d64ffb9d39d71c4baec116'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:45.433261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9fad83651d64ffb9d39d71c4baec116'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:45.433877 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9fad83651d64ffb9d39d71c4baec116'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:45.446178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9fad83651d64ffb9d39d71c4baec116'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:45.454567 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9fad83651d64ffb9d39d71c4baec116'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:45.468306 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1320/2653] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:45 2022] DELETE /identity/v3/users/e9fad83651d64ffb9d39d71c4baec116 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:45.472940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e69dcb-4984-4dda-93d3-1fd2251eb276 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:45.473422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e69dcb-4984-4dda-93d3-1fd2251eb276 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:45.473534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e69dcb-4984-4dda-93d3-1fd2251eb276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:45.490579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-13e69dcb-4984-4dda-93d3-1fd2251eb276 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:45.491912 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-13e69dcb-4984-4dda-93d3-1fd2251eb276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:45.565135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1334/2654] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:45.575192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:45.577227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:45.634586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdbc87c-46ea-4e64-86c8-bb033a7be288 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:45.638709 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c6253-f160-48f3-a5b9-d0647ed8a0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:45.639346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c6253-f160-48f3-a5b9-d0647ed8a0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:45.639630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c6253-f160-48f3-a5b9-d0647ed8a0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:45.639905 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242c6253-f160-48f3-a5b9-d0647ed8a0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:45.648802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c6253-f160-48f3-a5b9-d0647ed8a0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:45.650255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242c6253-f160-48f3-a5b9-d0647ed8a0f8 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:45.659486 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1321/2655] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:45.921910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e69dcb-4984-4dda-93d3-1fd2251eb276 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:45.923369 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-13e69dcb-4984-4dda-93d3-1fd2251eb276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:45.938471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e69dcb-4984-4dda-93d3-1fd2251eb276 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:45.943140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:45.944171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:45.944654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:45.944997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] PATH_INFO: `/v3/projects/12792c1e562b48e5bc59f61b8c1c10f8` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:45.946444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12792c1e562b48e5bc59f61b8c1c10f8)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:45.948781 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:45.993869 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12792c1e562b48e5bc59f61b8c1c10f8', 'request_id': 'req-202ff062-4ac7-4a72-848a-215a42cd3574'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.000161 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12792c1e562b48e5bc59f61b8c1c10f8', 'request_id': 'req-202ff062-4ac7-4a72-848a-215a42cd3574'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.007633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12792c1e562b48e5bc59f61b8c1c10f8', 'request_id': 'req-202ff062-4ac7-4a72-848a-215a42cd3574'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.018924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] The token cache is being invalidate because project 12792c1e562b48e5bc59f61b8c1c10f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:46.019208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.019820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.021334 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1335/2656] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:45 2022] DELETE /identity/v3/projects/12792c1e562b48e5bc59f61b8c1c10f8 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:46.025476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c6253-f160-48f3-a5b9-d0647ed8a0f8 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:46.034873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242c6253-f160-48f3-a5b9-d0647ed8a0f8 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:46.052893 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:46.092656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:46.093515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:46.093795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:46.094092 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] PATH_INFO: `/v3/users/c4231725cd854a49b45fa339f8dc81a4` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:46.095265 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4231725cd854a49b45fa339f8dc81a4)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:46.097161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:46.140411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4231725cd854a49b45fa339f8dc81a4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.141079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4231725cd854a49b45fa339f8dc81a4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.141686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4231725cd854a49b45fa339f8dc81a4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.153265 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4231725cd854a49b45fa339f8dc81a4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.160992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-dbba4375-245c-4081-a42f-b58e1e953d20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4231725cd854a49b45fa339f8dc81a4'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.173361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1322/2657] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:46 2022] DELETE /identity/v3/users/c4231725cd854a49b45fa339f8dc81a4 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:46.442868 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:46.444283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:46.459279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ff062-4ac7-4a72-848a-215a42cd3574 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:46.517821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:46.518734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:46.519028 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:46.519338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] PATH_INFO: `/v3/projects/85933743b9fa4d28a9f5c876bd958242` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:46.520479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85933743b9fa4d28a9f5c876bd958242)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:46.522431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:46.554747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:46.580016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85933743b9fa4d28a9f5c876bd958242', 'request_id': 'req-99c4da48-71e9-44fb-8179-180d321c57e5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.585748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85933743b9fa4d28a9f5c876bd958242', 'request_id': 'req-99c4da48-71e9-44fb-8179-180d321c57e5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.594506 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85933743b9fa4d28a9f5c876bd958242', 'request_id': 'req-99c4da48-71e9-44fb-8179-180d321c57e5'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.605651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] The token cache is being invalidate because project 85933743b9fa4d28a9f5c876bd958242 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:46.605943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.606566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-99c4da48-71e9-44fb-8179-180d321c57e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:46.608100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1336/2658] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:46 2022] DELETE /identity/v3/projects/85933743b9fa4d28a9f5c876bd958242 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:46.612917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32f7d1-e5f3-4f9f-bda3-44233b17eac8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:46.613376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32f7d1-e5f3-4f9f-bda3-44233b17eac8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:46.613497 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f32f7d1-e5f3-4f9f-bda3-44233b17eac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:46.630869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-8f32f7d1-e5f3-4f9f-bda3-44233b17eac8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:46.632072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-8f32f7d1-e5f3-4f9f-bda3-44233b17eac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:46.693017 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1323/2659] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:46.701187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be32b53-76ed-431e-9db7-09b42604337a None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:46.701474 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be32b53-76ed-431e-9db7-09b42604337a None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:46.701751 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be32b53-76ed-431e-9db7-09b42604337a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:46.721185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-3be32b53-76ed-431e-9db7-09b42604337a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:46.722436 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3be32b53-76ed-431e-9db7-09b42604337a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:46.730311 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1337/2660] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:46.734381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32f7d1-e5f3-4f9f-bda3-44233b17eac8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:46.743880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f32f7d1-e5f3-4f9f-bda3-44233b17eac8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:46.748380 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4d626-d8df-4468-8eab-be1ee6a2d6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:46.749028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4d626-d8df-4468-8eab-be1ee6a2d6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:46.749308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4d626-d8df-4468-8eab-be1ee6a2d6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:46.749582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a4d626-d8df-4468-8eab-be1ee6a2d6d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:46.750330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4d626-d8df-4468-8eab-be1ee6a2d6d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:15:46.750641 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4d626-d8df-4468-8eab-be1ee6a2d6d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:46.752203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a4d626-d8df-4468-8eab-be1ee6a2d6d7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:46.764147 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1324/2661] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:15:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:15:46.768397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be32b53-76ed-431e-9db7-09b42604337a None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:46.777397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be32b53-76ed-431e-9db7-09b42604337a None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:46.782645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94623f8-5296-47a6-892d-d68a99c1f8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:46.783287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94623f8-5296-47a6-892d-d68a99c1f8cc admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:46.783571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94623f8-5296-47a6-892d-d68a99c1f8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:46.783845 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94623f8-5296-47a6-892d-d68a99c1f8cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:46.784522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94623f8-5296-47a6-892d-d68a99c1f8cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:46.786432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94623f8-5296-47a6-892d-d68a99c1f8cc admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:46.810875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1338/2662] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:46 2022] POST /identity/v3/projects => generated 385 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:46.814886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4d626-d8df-4468-8eab-be1ee6a2d6d7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:46.824673 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a4d626-d8df-4468-8eab-be1ee6a2d6d7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:46.850662 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:46.882951 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780424b-cd4a-4c16-b437-669351a51164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:46.883625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780424b-cd4a-4c16-b437-669351a51164 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:46.883915 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780424b-cd4a-4c16-b437-669351a51164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:46.884199 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780424b-cd4a-4c16-b437-669351a51164 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:46.884897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780424b-cd4a-4c16-b437-669351a51164 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:46.886854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780424b-cd4a-4c16-b437-669351a51164 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:46.908554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1325/2663] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:46 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:46.912616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94623f8-5296-47a6-892d-d68a99c1f8cc admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:46.923262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94623f8-5296-47a6-892d-d68a99c1f8cc admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:46.927819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df43e5-3f1c-4724-b14e-11cccb7ddd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:46.928469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df43e5-3f1c-4724-b14e-11cccb7ddd5e admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:46.928751 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df43e5-3f1c-4724-b14e-11cccb7ddd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:46.929022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df43e5-3f1c-4724-b14e-11cccb7ddd5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:46.929739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df43e5-3f1c-4724-b14e-11cccb7ddd5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:46.931362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df43e5-3f1c-4724-b14e-11cccb7ddd5e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:46.942233 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1339/2664] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:46.945891 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780424b-cd4a-4c16-b437-669351a51164 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:46.956462 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780424b-cd4a-4c16-b437-669351a51164 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:46.961219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984d3a1-6ded-42b8-808f-be43f0aca70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:46.961911 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d3a1-6ded-42b8-808f-be43f0aca70a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:46.962237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d3a1-6ded-42b8-808f-be43f0aca70a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:46.962522 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984d3a1-6ded-42b8-808f-be43f0aca70a admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:46.963255 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984d3a1-6ded-42b8-808f-be43f0aca70a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:46.964880 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984d3a1-6ded-42b8-808f-be43f0aca70a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:46.975299 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1326/2665] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:46.979331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df43e5-3f1c-4724-b14e-11cccb7ddd5e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:46.989033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df43e5-3f1c-4724-b14e-11cccb7ddd5e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:46.993537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96a59f-4301-4b38-9105-c56239772b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:46.994441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96a59f-4301-4b38-9105-c56239772b0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:46.994723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96a59f-4301-4b38-9105-c56239772b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:46.994994 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96a59f-4301-4b38-9105-c56239772b0b admin admin] PATH_INFO: `/v3/projects/4491c4633c88478e98ddda5630bba8d4/users/0765cd8c7f0d40ffb1bbf524854ff73f/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:47.009932 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96a59f-4301-4b38-9105-c56239772b0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4491c4633c88478e98ddda5630bba8d4, user_id=0765cd8c7f0d40ffb1bbf524854ff73f, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:47.012640 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96a59f-4301-4b38-9105-c56239772b0b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:47.025728 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1340/2666] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:46 2022] PUT /identity/v3/projects/4491c4633c88478e98ddda5630bba8d4/users/0765cd8c7f0d40ffb1bbf524854ff73f/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:47.036559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3c62a-9356-4a63-b94f-e1f951c62c6e None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:47.036855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3c62a-9356-4a63-b94f-e1f951c62c6e None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:47.037137 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa3c62a-9356-4a63-b94f-e1f951c62c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:47.063260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-4aa3c62a-9356-4a63-b94f-e1f951c62c6e None None] MFA Rules not processed for user `0765cd8c7f0d40ffb1bbf524854ff73f`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:47.064470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-4aa3c62a-9356-4a63-b94f-e1f951c62c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:47.127676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1327/2667] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:47.132529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96a59f-4301-4b38-9105-c56239772b0b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:47.194762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96a59f-4301-4b38-9105-c56239772b0b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:47.199520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87b0b7-79e9-412a-81af-c62edacee55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:47.200177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b0b7-79e9-412a-81af-c62edacee55e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:47.200455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b0b7-79e9-412a-81af-c62edacee55e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:47.201371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e87b0b7-79e9-412a-81af-c62edacee55e admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:47.204081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87b0b7-79e9-412a-81af-c62edacee55e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:47.204081 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e87b0b7-79e9-412a-81af-c62edacee55e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:47.238809 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1341/2668] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:47 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:47.239996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3c62a-9356-4a63-b94f-e1f951c62c6e None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:47.251322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa3c62a-9356-4a63-b94f-e1f951c62c6e None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:47.308635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a64552-45e3-4d8c-914a-1c77ff2789d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:47.309313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a64552-45e3-4d8c-914a-1c77ff2789d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:47.309597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a64552-45e3-4d8c-914a-1c77ff2789d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:47.309918 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a64552-45e3-4d8c-914a-1c77ff2789d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:47.311141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a64552-45e3-4d8c-914a-1c77ff2789d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:47.313037 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a64552-45e3-4d8c-914a-1c77ff2789d9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:47.333087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1328/2669] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:47 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:47.337208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87b0b7-79e9-412a-81af-c62edacee55e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:47.348122 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e87b0b7-79e9-412a-81af-c62edacee55e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:47.352741 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d2483-a180-42ec-a423-7e4fa4c74eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:47.353402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d2483-a180-42ec-a423-7e4fa4c74eff admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:47.353688 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d2483-a180-42ec-a423-7e4fa4c74eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:47.353969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d2483-a180-42ec-a423-7e4fa4c74eff admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:47.354721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d2483-a180-42ec-a423-7e4fa4c74eff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:47.356307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9d2483-a180-42ec-a423-7e4fa4c74eff admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:47.367924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1342/2670] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:15:47.372065 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a64552-45e3-4d8c-914a-1c77ff2789d9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:47.383402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a64552-45e3-4d8c-914a-1c77ff2789d9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:47.388154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcad65f-a3e4-44b1-9dea-685a9edb78d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:47.389110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcad65f-a3e4-44b1-9dea-685a9edb78d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:47.389471 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcad65f-a3e4-44b1-9dea-685a9edb78d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:47.389820 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcad65f-a3e4-44b1-9dea-685a9edb78d7 admin admin] PATH_INFO: `/v3/projects/c2803fa862404721832bb61575e47fe4/users/550fc5db037440939d406fa82480ee6b/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:47.407233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcad65f-a3e4-44b1-9dea-685a9edb78d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2803fa862404721832bb61575e47fe4, user_id=550fc5db037440939d406fa82480ee6b, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:47.409894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcad65f-a3e4-44b1-9dea-685a9edb78d7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:47.423351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1329/2671] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:47 2022] PUT /identity/v3/projects/c2803fa862404721832bb61575e47fe4/users/550fc5db037440939d406fa82480ee6b/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:47.427292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d2483-a180-42ec-a423-7e4fa4c74eff admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:47.437795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d2483-a180-42ec-a423-7e4fa4c74eff admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:47.456993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:47.494408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5fde6-b375-4ff8-983d-4b02ad4c7dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:47.495071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5fde6-b375-4ff8-983d-4b02ad4c7dc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:47.495359 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5fde6-b375-4ff8-983d-4b02ad4c7dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:47.495641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5fde6-b375-4ff8-983d-4b02ad4c7dc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:47.496361 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5fde6-b375-4ff8-983d-4b02ad4c7dc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:47.498608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5fde6-b375-4ff8-983d-4b02ad4c7dc9 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:47.508636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1343/2672] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:47.512863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcad65f-a3e4-44b1-9dea-685a9edb78d7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:47.518343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:47.524434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcad65f-a3e4-44b1-9dea-685a9edb78d7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:47.529309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639484a-0640-41e5-8c8e-264516ffa6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:47.530209 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639484a-0640-41e5-8c8e-264516ffa6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:47.530497 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639484a-0640-41e5-8c8e-264516ffa6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:47.530774 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5639484a-0640-41e5-8c8e-264516ffa6e9 admin admin] PATH_INFO: `/v3/projects/c2803fa862404721832bb61575e47fe4/users/550fc5db037440939d406fa82480ee6b/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:47.532740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639484a-0640-41e5-8c8e-264516ffa6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2803fa862404721832bb61575e47fe4, user_id=550fc5db037440939d406fa82480ee6b, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:47.535408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5639484a-0640-41e5-8c8e-264516ffa6e9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:47.545895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1330/2673] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:47 2022] PUT /identity/v3/projects/c2803fa862404721832bb61575e47fe4/users/550fc5db037440939d406fa82480ee6b/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:47.555673 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35af62-1def-4667-91a2-18e049f3ef58 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:47.556145 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35af62-1def-4667-91a2-18e049f3ef58 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:47.556360 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35af62-1def-4667-91a2-18e049f3ef58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:47.582354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4f35af62-1def-4667-91a2-18e049f3ef58 None None] MFA Rules not processed for user `550fc5db037440939d406fa82480ee6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:47.583562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4f35af62-1def-4667-91a2-18e049f3ef58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:47.656964 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1344/2674] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:47.667342 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639484a-0640-41e5-8c8e-264516ffa6e9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:47.669078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5639484a-0640-41e5-8c8e-264516ffa6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:47.726927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5639484a-0640-41e5-8c8e-264516ffa6e9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:47.730850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f1069-5cbb-4ad8-830b-d6f2f2677869 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:47.731428 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1069-5cbb-4ad8-830b-d6f2f2677869 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:47.731748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1069-5cbb-4ad8-830b-d6f2f2677869 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:47.732621 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f1069-5cbb-4ad8-830b-d6f2f2677869 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:47.741490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f1069-5cbb-4ad8-830b-d6f2f2677869 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:47.742944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1f1069-5cbb-4ad8-830b-d6f2f2677869 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:47.752283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1331/2675] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:47 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:48.271457 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35af62-1def-4667-91a2-18e049f3ef58 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:48.333309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35af62-1def-4667-91a2-18e049f3ef58 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:48.337819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:48.338718 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:48.339004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:48.339296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] PATH_INFO: `/v3/users/0765cd8c7f0d40ffb1bbf524854ff73f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:48.340473 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0765cd8c7f0d40ffb1bbf524854ff73f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:48.343077 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:48.391382 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0765cd8c7f0d40ffb1bbf524854ff73f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:48.392042 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0765cd8c7f0d40ffb1bbf524854ff73f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:48.393242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:48.400659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0765cd8c7f0d40ffb1bbf524854ff73f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:48.408662 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0765cd8c7f0d40ffb1bbf524854ff73f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:48.409275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0765cd8c7f0d40ffb1bbf524854ff73f'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:48.426380 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1345/2676] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:48 2022] DELETE /identity/v3/users/0765cd8c7f0d40ffb1bbf524854ff73f => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:48.430777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b390fbb-6e2e-4f16-8d7e-8bc8ab94d70a None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:48.431074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b390fbb-6e2e-4f16-8d7e-8bc8ab94d70a None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:48.431356 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b390fbb-6e2e-4f16-8d7e-8bc8ab94d70a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:48.449159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2b390fbb-6e2e-4f16-8d7e-8bc8ab94d70a None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:48.450419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2b390fbb-6e2e-4f16-8d7e-8bc8ab94d70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:48.507663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1332/2677] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:48.518019 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:48.519461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:48.574701 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab18656-67a6-439a-bc8f-5fb95ed7da6e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:48.579999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c95b7e-69f8-4010-b373-71772f1a7172 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:48.579999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c95b7e-69f8-4010-b373-71772f1a7172 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:48.579999 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c95b7e-69f8-4010-b373-71772f1a7172 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:48.580348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c95b7e-69f8-4010-b373-71772f1a7172 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:48.589126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c95b7e-69f8-4010-b373-71772f1a7172 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:48.590595 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c95b7e-69f8-4010-b373-71772f1a7172 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:48.599858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1346/2678] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:48.872482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b390fbb-6e2e-4f16-8d7e-8bc8ab94d70a None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:48.874456 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2b390fbb-6e2e-4f16-8d7e-8bc8ab94d70a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:48.887805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b390fbb-6e2e-4f16-8d7e-8bc8ab94d70a None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:48.892423 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:48.893301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:48.893570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:48.893846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] PATH_INFO: `/v3/projects/4491c4633c88478e98ddda5630bba8d4` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:48.895078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4491c4633c88478e98ddda5630bba8d4)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:48.897087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:48.944949 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4491c4633c88478e98ddda5630bba8d4', 'request_id': 'req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:48.952071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4491c4633c88478e98ddda5630bba8d4', 'request_id': 'req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:48.957635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4491c4633c88478e98ddda5630bba8d4', 'request_id': 'req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:48.968226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] The token cache is being invalidate because project 4491c4633c88478e98ddda5630bba8d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:48.968508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:48.969124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:48.970667 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1333/2679] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:48 2022] DELETE /identity/v3/projects/4491c4633c88478e98ddda5630bba8d4 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:48.974583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c95b7e-69f8-4010-b373-71772f1a7172 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:48.984484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c95b7e-69f8-4010-b373-71772f1a7172 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:49.043202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:49.044061 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.044827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.045100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] PATH_INFO: `/v3/users/550fc5db037440939d406fa82480ee6b` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:49.046283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=550fc5db037440939d406fa82480ee6b)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:49.048169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:49.091789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '550fc5db037440939d406fa82480ee6b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:49.092420 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '550fc5db037440939d406fa82480ee6b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:49.099024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '550fc5db037440939d406fa82480ee6b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:49.105906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '550fc5db037440939d406fa82480ee6b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:49.106574 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-01b3ffc7-70ff-48f5-a71c-e205cd6d8980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '550fc5db037440939d406fa82480ee6b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:49.112355 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:49.126508 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1347/2680] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:48 2022] DELETE /identity/v3/users/550fc5db037440939d406fa82480ee6b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:49.399406 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:49.400819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:49.414832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff96026-cfc0-4c16-bef5-cda4c8b3b3b9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:49.430609 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:49.476362 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:49.477148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.477427 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.477697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] PATH_INFO: `/v3/projects/c2803fa862404721832bb61575e47fe4` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:49.478908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2803fa862404721832bb61575e47fe4)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:49.480808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:49.525627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2803fa862404721832bb61575e47fe4', 'request_id': 'req-59deb8b5-ca02-4f89-8c98-07af619a9252'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:49.533071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2803fa862404721832bb61575e47fe4', 'request_id': 'req-59deb8b5-ca02-4f89-8c98-07af619a9252'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:49.538495 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2803fa862404721832bb61575e47fe4', 'request_id': 'req-59deb8b5-ca02-4f89-8c98-07af619a9252'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:49.549580 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] The token cache is being invalidate because project c2803fa862404721832bb61575e47fe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:49.549866 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:49.550528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-59deb8b5-ca02-4f89-8c98-07af619a9252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:49.552034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1334/2681] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:49 2022] DELETE /identity/v3/projects/c2803fa862404721832bb61575e47fe4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:49.556917 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606496a4-02cd-49da-975d-f72555b29163 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.557293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606496a4-02cd-49da-975d-f72555b29163 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.557600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606496a4-02cd-49da-975d-f72555b29163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:49.574752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-606496a4-02cd-49da-975d-f72555b29163 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:49.576499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-606496a4-02cd-49da-975d-f72555b29163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:49.635201 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1348/2682] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:49.643528 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396db79-24cf-4f2e-a7db-548f7151b75d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.643818 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396db79-24cf-4f2e-a7db-548f7151b75d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.644097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396db79-24cf-4f2e-a7db-548f7151b75d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:49.662927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d396db79-24cf-4f2e-a7db-548f7151b75d None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:49.664125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d396db79-24cf-4f2e-a7db-548f7151b75d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:49.672177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1335/2683] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:49.676242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606496a4-02cd-49da-975d-f72555b29163 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:49.685793 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606496a4-02cd-49da-975d-f72555b29163 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:49.690293 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041a025-f582-4b9b-8502-b362fea46c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:49.690941 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041a025-f582-4b9b-8502-b362fea46c43 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.691223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041a025-f582-4b9b-8502-b362fea46c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.691497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041a025-f582-4b9b-8502-b362fea46c43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:49.692214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041a025-f582-4b9b-8502-b362fea46c43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:15:49.692523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041a025-f582-4b9b-8502-b362fea46c43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:49.694133 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041a025-f582-4b9b-8502-b362fea46c43 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:49.705800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1349/2684] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:15:49.709959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396db79-24cf-4f2e-a7db-548f7151b75d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:49.720690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d396db79-24cf-4f2e-a7db-548f7151b75d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:49.725804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91544e-dfcf-46f1-adfa-942249db13a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:49.726527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91544e-dfcf-46f1-adfa-942249db13a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.726757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91544e-dfcf-46f1-adfa-942249db13a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.727029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91544e-dfcf-46f1-adfa-942249db13a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:49.727718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91544e-dfcf-46f1-adfa-942249db13a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:49.729590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91544e-dfcf-46f1-adfa-942249db13a9 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:49.753899 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1336/2685] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:49 2022] POST /identity/v3/projects => generated 359 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:49.759675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041a025-f582-4b9b-8502-b362fea46c43 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:49.770054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041a025-f582-4b9b-8502-b362fea46c43 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:49.824499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54b945-b298-4ac6-a55c-aa0e63cfc02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:49.825156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54b945-b298-4ac6-a55c-aa0e63cfc02b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.825929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54b945-b298-4ac6-a55c-aa0e63cfc02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.826242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c54b945-b298-4ac6-a55c-aa0e63cfc02b admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:49.826924 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54b945-b298-4ac6-a55c-aa0e63cfc02b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:49.828828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c54b945-b298-4ac6-a55c-aa0e63cfc02b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:49.848364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1350/2686] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:49 2022] POST /identity/v3/users => generated 280 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:49.852542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91544e-dfcf-46f1-adfa-942249db13a9 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:49.862728 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91544e-dfcf-46f1-adfa-942249db13a9 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:49.867195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec54462-2985-4002-b558-7c6e1fe8a42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:49.867869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54462-2985-4002-b558-7c6e1fe8a42f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.868154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54462-2985-4002-b558-7c6e1fe8a42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.868437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec54462-2985-4002-b558-7c6e1fe8a42f admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:49.869158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec54462-2985-4002-b558-7c6e1fe8a42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:49.870779 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec54462-2985-4002-b558-7c6e1fe8a42f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:49.881553 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1337/2687] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:49.885493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54b945-b298-4ac6-a55c-aa0e63cfc02b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:49.895893 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c54b945-b298-4ac6-a55c-aa0e63cfc02b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:49.900483 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add1a15b-c28c-4e0c-9a03-493dcec730c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:49.901132 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1a15b-c28c-4e0c-9a03-493dcec730c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.901414 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1a15b-c28c-4e0c-9a03-493dcec730c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.901691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add1a15b-c28c-4e0c-9a03-493dcec730c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:49.902443 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1a15b-c28c-4e0c-9a03-493dcec730c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:49.904649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add1a15b-c28c-4e0c-9a03-493dcec730c7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:49.914626 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1351/2688] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:49.918649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec54462-2985-4002-b558-7c6e1fe8a42f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:49.928690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec54462-2985-4002-b558-7c6e1fe8a42f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:49.933176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdd501-0a78-4f23-9b12-d4e17b92f139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:49.934069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdd501-0a78-4f23-9b12-d4e17b92f139 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.934353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdd501-0a78-4f23-9b12-d4e17b92f139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.934627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdd501-0a78-4f23-9b12-d4e17b92f139 admin admin] PATH_INFO: `/v3/projects/b6f51e2a5aa3420a92ec6bdfee55678c/users/ada2287746504d1a94e72c6f7241b559/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:49.949050 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdd501-0a78-4f23-9b12-d4e17b92f139 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f51e2a5aa3420a92ec6bdfee55678c, user_id=ada2287746504d1a94e72c6f7241b559, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:49.952329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdd501-0a78-4f23-9b12-d4e17b92f139 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:49.964885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1338/2689] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:49 2022] PUT /identity/v3/projects/b6f51e2a5aa3420a92ec6bdfee55678c/users/ada2287746504d1a94e72c6f7241b559/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:49.974736 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b4c03-71ff-469b-8c72-0f79a76b7a18 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:49.975033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b4c03-71ff-469b-8c72-0f79a76b7a18 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:49.975312 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b4c03-71ff-469b-8c72-0f79a76b7a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:50.001230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-194b4c03-71ff-469b-8c72-0f79a76b7a18 None None] MFA Rules not processed for user `ada2287746504d1a94e72c6f7241b559`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:50.002405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-194b4c03-71ff-469b-8c72-0f79a76b7a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:50.042585 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:50.066310 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1352/2690] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:49 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:50.097365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdd501-0a78-4f23-9b12-d4e17b92f139 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:50.099064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-60cdd501-0a78-4f23-9b12-d4e17b92f139 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:50.144932 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:50.158626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdd501-0a78-4f23-9b12-d4e17b92f139 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:50.162334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38e395-5b70-4b92-83e4-381baf1ff123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:50.162985 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38e395-5b70-4b92-83e4-381baf1ff123 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:50.163270 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38e395-5b70-4b92-83e4-381baf1ff123 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:50.163561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38e395-5b70-4b92-83e4-381baf1ff123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:50.172069 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38e395-5b70-4b92-83e4-381baf1ff123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:50.173516 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38e395-5b70-4b92-83e4-381baf1ff123 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:50.182430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1339/2691] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:50 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:50.196827 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b4c03-71ff-469b-8c72-0f79a76b7a18 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:50.258661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b4c03-71ff-469b-8c72-0f79a76b7a18 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:50.263790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:50.264654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:50.264937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:50.265208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] PATH_INFO: `/v3/users/ada2287746504d1a94e72c6f7241b559` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:50.266424 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ada2287746504d1a94e72c6f7241b559)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:50.268313 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:50.315799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ada2287746504d1a94e72c6f7241b559'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:50.316469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ada2287746504d1a94e72c6f7241b559'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:50.323226 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ada2287746504d1a94e72c6f7241b559'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:50.330239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ada2287746504d1a94e72c6f7241b559'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:50.330863 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ada2287746504d1a94e72c6f7241b559'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:50.347524 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1353/2692] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:50 2022] DELETE /identity/v3/users/ada2287746504d1a94e72c6f7241b559 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:50.351972 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a14b3-15d3-4e2e-b09e-9944e47b40b6 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:50.352263 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a14b3-15d3-4e2e-b09e-9944e47b40b6 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:50.352571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a14b3-15d3-4e2e-b09e-9944e47b40b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:50.369610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c00a14b3-15d3-4e2e-b09e-9944e47b40b6 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:50.370854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c00a14b3-15d3-4e2e-b09e-9944e47b40b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:50.427934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1340/2693] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:50.438126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:50.439497 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:50.495100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d6c51-fa5f-46a8-82b0-945576087782 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:50.499502 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51afb254-ee98-44b4-b378-a350613fdbb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:50.500140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afb254-ee98-44b4-b378-a350613fdbb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:50.500422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afb254-ee98-44b4-b378-a350613fdbb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:50.500697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51afb254-ee98-44b4-b378-a350613fdbb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:50.509822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afb254-ee98-44b4-b378-a350613fdbb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:50.511221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51afb254-ee98-44b4-b378-a350613fdbb0 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:50.520379 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1354/2694] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:50.795658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a14b3-15d3-4e2e-b09e-9944e47b40b6 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:50.797414 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c00a14b3-15d3-4e2e-b09e-9944e47b40b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:50.811402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00a14b3-15d3-4e2e-b09e-9944e47b40b6 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:50.816909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:50.817967 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:50.818371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:50.818739 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] PATH_INFO: `/v3/projects/b6f51e2a5aa3420a92ec6bdfee55678c` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:50.820924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f51e2a5aa3420a92ec6bdfee55678c)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:50.823472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:50.877273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f51e2a5aa3420a92ec6bdfee55678c', 'request_id': 'req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:50.884908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f51e2a5aa3420a92ec6bdfee55678c', 'request_id': 'req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:50.890764 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f51e2a5aa3420a92ec6bdfee55678c', 'request_id': 'req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:50.901649 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] The token cache is being invalidate because project b6f51e2a5aa3420a92ec6bdfee55678c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:50.901939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:50.902596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cdeb4cf1-aae1-4dc1-a33f-dbb6ff062293 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:50.904658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1341/2695] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:50 2022] DELETE /identity/v3/projects/b6f51e2a5aa3420a92ec6bdfee55678c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:50.909752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2924fdd-3f05-43db-814c-64a5286e6a3b None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:50.910297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2924fdd-3f05-43db-814c-64a5286e6a3b None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:50.910499 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2924fdd-3f05-43db-814c-64a5286e6a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:50.917599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:50.930716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c2924fdd-3f05-43db-814c-64a5286e6a3b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:50.931916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c2924fdd-3f05-43db-814c-64a5286e6a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:50.991018 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1355/2696] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:50.998994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf49af4-b36d-4260-bcf7-7142ad3ac0f4 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:50.999352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf49af4-b36d-4260-bcf7-7142ad3ac0f4 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:50.999573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf49af4-b36d-4260-bcf7-7142ad3ac0f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:51.018009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-aaf49af4-b36d-4260-bcf7-7142ad3ac0f4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:51.019215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-aaf49af4-b36d-4260-bcf7-7142ad3ac0f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:51.027370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1342/2697] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:51.032405 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2924fdd-3f05-43db-814c-64a5286e6a3b None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:51.042759 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2924fdd-3f05-43db-814c-64a5286e6a3b None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:51.048079 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2baa1c-b1d8-4393-90f3-12e1c139e94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:51.048772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2baa1c-b1d8-4393-90f3-12e1c139e94c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:51.049054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2baa1c-b1d8-4393-90f3-12e1c139e94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:51.049328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2baa1c-b1d8-4393-90f3-12e1c139e94c admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:51.050110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2baa1c-b1d8-4393-90f3-12e1c139e94c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:15:51.050421 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2baa1c-b1d8-4393-90f3-12e1c139e94c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:51.052015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2baa1c-b1d8-4393-90f3-12e1c139e94c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:51.064982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1356/2698] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:15:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:15:51.069799 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf49af4-b36d-4260-bcf7-7142ad3ac0f4 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:51.079814 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf49af4-b36d-4260-bcf7-7142ad3ac0f4 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:51.084559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64703172-fbde-41cb-ab9a-3b5829baabda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:51.085226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64703172-fbde-41cb-ab9a-3b5829baabda admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:51.085506 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64703172-fbde-41cb-ab9a-3b5829baabda admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:51.085777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64703172-fbde-41cb-ab9a-3b5829baabda admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:51.086493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64703172-fbde-41cb-ab9a-3b5829baabda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:51.088396 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64703172-fbde-41cb-ab9a-3b5829baabda admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:51.113371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1343/2699] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:51 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:51.117409 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2baa1c-b1d8-4393-90f3-12e1c139e94c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:51.127190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2baa1c-b1d8-4393-90f3-12e1c139e94c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:51.182649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857761ac-f754-41a5-b71f-0e90f5797fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:51.183310 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857761ac-f754-41a5-b71f-0e90f5797fce admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:51.183591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857761ac-f754-41a5-b71f-0e90f5797fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:51.183864 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857761ac-f754-41a5-b71f-0e90f5797fce admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:51.184545 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857761ac-f754-41a5-b71f-0e90f5797fce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:51.186471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857761ac-f754-41a5-b71f-0e90f5797fce admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:51.206250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1357/2700] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:51 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:51.210418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64703172-fbde-41cb-ab9a-3b5829baabda admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:51.221164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64703172-fbde-41cb-ab9a-3b5829baabda admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:51.225658 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e7f54-4a4f-4c7d-abeb-dd3bba751875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:51.226339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e7f54-4a4f-4c7d-abeb-dd3bba751875 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:51.226653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e7f54-4a4f-4c7d-abeb-dd3bba751875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:51.226928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e7f54-4a4f-4c7d-abeb-dd3bba751875 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:51.227645 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e7f54-4a4f-4c7d-abeb-dd3bba751875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:51.229225 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e7f54-4a4f-4c7d-abeb-dd3bba751875 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:51.239608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1344/2701] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:51.243323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857761ac-f754-41a5-b71f-0e90f5797fce admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:51.253046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857761ac-f754-41a5-b71f-0e90f5797fce admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:51.257811 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ba6736-86a7-4bcb-8b72-862c38d343b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:51.258482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba6736-86a7-4bcb-8b72-862c38d343b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:51.258766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba6736-86a7-4bcb-8b72-862c38d343b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:51.259040 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ba6736-86a7-4bcb-8b72-862c38d343b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:51.259747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba6736-86a7-4bcb-8b72-862c38d343b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:51.261322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ba6736-86a7-4bcb-8b72-862c38d343b1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:51.273504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1358/2702] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:51.277314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e7f54-4a4f-4c7d-abeb-dd3bba751875 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:51.287882 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e7f54-4a4f-4c7d-abeb-dd3bba751875 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:51.292345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41ca45-1437-4ecc-9d28-3903809a5481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:51.293297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ca45-1437-4ecc-9d28-3903809a5481 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:51.293578 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ca45-1437-4ecc-9d28-3903809a5481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:51.293846 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c41ca45-1437-4ecc-9d28-3903809a5481 admin admin] PATH_INFO: `/v3/projects/1e6c73bfd5fa4810863047798b6c03da/users/8f84cfa797de4d31b48d15461e9507f8/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:51.307900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41ca45-1437-4ecc-9d28-3903809a5481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e6c73bfd5fa4810863047798b6c03da, user_id=8f84cfa797de4d31b48d15461e9507f8, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:51.310533 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c41ca45-1437-4ecc-9d28-3903809a5481 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:51.323607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1345/2703] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:51 2022] PUT /identity/v3/projects/1e6c73bfd5fa4810863047798b6c03da/users/8f84cfa797de4d31b48d15461e9507f8/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:51.334979 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e48d8-14cc-441e-bfa7-62d34ab1dbad None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:51.335264 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e48d8-14cc-441e-bfa7-62d34ab1dbad None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:51.335532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e48d8-14cc-441e-bfa7-62d34ab1dbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:51.360790 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-234e48d8-14cc-441e-bfa7-62d34ab1dbad None None] MFA Rules not processed for user `8f84cfa797de4d31b48d15461e9507f8`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:51.362097 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-234e48d8-14cc-441e-bfa7-62d34ab1dbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:51.422223 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:51.426093 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1359/2704] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:51 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:51.437235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41ca45-1437-4ecc-9d28-3903809a5481 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:51.438944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6c41ca45-1437-4ecc-9d28-3903809a5481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:51.445385 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:51.498400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c41ca45-1437-4ecc-9d28-3903809a5481 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:51.502058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c327c-1048-4617-9db7-e90843501dff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:51.502708 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c327c-1048-4617-9db7-e90843501dff service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:51.502992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c327c-1048-4617-9db7-e90843501dff service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:51.503265 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c327c-1048-4617-9db7-e90843501dff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:51.511726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c327c-1048-4617-9db7-e90843501dff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:51.513737 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541c327c-1048-4617-9db7-e90843501dff service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:51.522384 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1346/2705] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:54.458882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e48d8-14cc-441e-bfa7-62d34ab1dbad None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:54.460281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-234e48d8-14cc-441e-bfa7-62d34ab1dbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:54.477245 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c327c-1048-4617-9db7-e90843501dff service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:54.529134 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e48d8-14cc-441e-bfa7-62d34ab1dbad None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:54.533589 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:54.534479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:54.534768 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:54.535044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] PATH_INFO: `/v3/users/3a27b452cbee46cdb7a37752f554527d` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:54.536304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a27b452cbee46cdb7a37752f554527d)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:54.538242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:54.540281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541c327c-1048-4617-9db7-e90843501dff service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:54.545233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:54.546136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:54.546425 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:54.546700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] PATH_INFO: `/v3/users/8f84cfa797de4d31b48d15461e9507f8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:54.547896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f84cfa797de4d31b48d15461e9507f8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:54.549798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:54.588045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a27b452cbee46cdb7a37752f554527d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:54.588733 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a27b452cbee46cdb7a37752f554527d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:54.596705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a27b452cbee46cdb7a37752f554527d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:54.603772 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a27b452cbee46cdb7a37752f554527d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:54.604264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f84cfa797de4d31b48d15461e9507f8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:54.604554 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-f1ff8262-4040-4456-b23b-24915316b68a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a27b452cbee46cdb7a37752f554527d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:54.604914 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f84cfa797de4d31b48d15461e9507f8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:54.605509 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f84cfa797de4d31b48d15461e9507f8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:54.618201 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f84cfa797de4d31b48d15461e9507f8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:54.623978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1360/2706] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:54 2022] DELETE /identity/v3/users/3a27b452cbee46cdb7a37752f554527d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:54.626311 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-e1432994-4bdc-4b2b-a761-de418a3b2071 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f84cfa797de4d31b48d15461e9507f8'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:54.628579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38964e3e-e5bd-4023-905f-d9690af6c2ef None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:54.628871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38964e3e-e5bd-4023-905f-d9690af6c2ef None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:54.629404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38964e3e-e5bd-4023-905f-d9690af6c2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:54.640829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1347/2707] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:54 2022] DELETE /identity/v3/users/8f84cfa797de4d31b48d15461e9507f8 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:54.645067 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e181939-254d-44e2-b4be-cd6c0af3bdb3 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:54.645361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e181939-254d-44e2-b4be-cd6c0af3bdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:54.645644 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e181939-254d-44e2-b4be-cd6c0af3bdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:54.649158 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-38964e3e-e5bd-4023-905f-d9690af6c2ef None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:54.650386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-38964e3e-e5bd-4023-905f-d9690af6c2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:54.662327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-1e181939-254d-44e2-b4be-cd6c0af3bdb3 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:54.663512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1e181939-254d-44e2-b4be-cd6c0af3bdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:54.685863 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:54.711600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1361/2708] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:54.722465 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38964e3e-e5bd-4023-905f-d9690af6c2ef None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:54.723821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-38964e3e-e5bd-4023-905f-d9690af6c2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:54.727075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1348/2709] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:15:54.740231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e181939-254d-44e2-b4be-cd6c0af3bdb3 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:54.741590 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1e181939-254d-44e2-b4be-cd6c0af3bdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:54.745244 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:54.783043 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38964e3e-e5bd-4023-905f-d9690af6c2ef None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:54.786825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8ddf3-ebb3-4360-83d7-230331c33855 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:54.787479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8ddf3-ebb3-4360-83d7-230331c33855 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:54.787766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8ddf3-ebb3-4360-83d7-230331c33855 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:54.788046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8ddf3-ebb3-4360-83d7-230331c33855 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:54.797787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8ddf3-ebb3-4360-83d7-230331c33855 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:54.799348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8ddf3-ebb3-4360-83d7-230331c33855 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:54.800447 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e181939-254d-44e2-b4be-cd6c0af3bdb3 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:54.804214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfdec1-565b-409c-b154-98a4f5461755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:54.804853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfdec1-565b-409c-b154-98a4f5461755 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:54.805139 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfdec1-565b-409c-b154-98a4f5461755 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:54.805411 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfdec1-565b-409c-b154-98a4f5461755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:54.808963 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1362/2710] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:15:54.815493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfdec1-565b-409c-b154-98a4f5461755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:54.816933 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfdec1-565b-409c-b154-98a4f5461755 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:54.826561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1349/2711] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:15:55.168171 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8ddf3-ebb3-4360-83d7-230331c33855 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.170594 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c0b8ddf3-ebb3-4360-83d7-230331c33855 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:55.186336 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8ddf3-ebb3-4360-83d7-230331c33855 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:55.192496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:55.193270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.193639 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.193825 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] PATH_INFO: `/v3/projects/1e471a5e4aa948af9466c062b550d035` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.195190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e471a5e4aa948af9466c062b550d035)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:55.197316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:55.205796 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfdec1-565b-409c-b154-98a4f5461755 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.207503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-36cfdec1-565b-409c-b154-98a4f5461755 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:55.222783 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfdec1-565b-409c-b154-98a4f5461755 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:55.228073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:55.228947 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.229308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.229587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] PATH_INFO: `/v3/projects/1e6c73bfd5fa4810863047798b6c03da` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.231035 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e6c73bfd5fa4810863047798b6c03da)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:55.233381 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:55.252860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e471a5e4aa948af9466c062b550d035', 'request_id': 'req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.260049 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e471a5e4aa948af9466c062b550d035', 'request_id': 'req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.267813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e471a5e4aa948af9466c062b550d035', 'request_id': 'req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.279418 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] The token cache is being invalidate because project 1e471a5e4aa948af9466c062b550d035 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:55.279708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.280356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.281920 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1363/2712] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:55 2022] DELETE /identity/v3/projects/1e471a5e4aa948af9466c062b550d035 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:55.282703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e6c73bfd5fa4810863047798b6c03da', 'request_id': 'req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.285666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.287000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:55.290387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e6c73bfd5fa4810863047798b6c03da', 'request_id': 'req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.296765 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e6c73bfd5fa4810863047798b6c03da', 'request_id': 'req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.308022 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] The token cache is being invalidate because project 1e6c73bfd5fa4810863047798b6c03da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:55.308338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.308974 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-b49f411b-e71b-4f4f-9a69-86c9a5dcff23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.310560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1350/2713] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:55 2022] DELETE /identity/v3/projects/1e6c73bfd5fa4810863047798b6c03da => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:55.315339 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d4789-4499-4053-9df2-c98b45e64a86 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.315627 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d4789-4499-4053-9df2-c98b45e64a86 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.315909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22d4789-4499-4053-9df2-c98b45e64a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.335941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b22d4789-4499-4053-9df2-c98b45e64a86 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:55.337173 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b22d4789-4499-4053-9df2-c98b45e64a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:55.354271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7fe4ed-cfa7-4266-978f-04d9779cc51d admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:55.390218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:55.404613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:55.411188 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1351/2714] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:15:55.412927 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfa7b7-e158-4c24-83fc-1378bc2afeb8 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.413191 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfa7b7-e158-4c24-83fc-1378bc2afeb8 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.413664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dfa7b7-e158-4c24-83fc-1378bc2afeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.416395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:55.417267 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.417543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.417817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] PATH_INFO: `/v3/users/9b9b54f1a5cc4d48a6369a50a611fdb9` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.419022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b9b54f1a5cc4d48a6369a50a611fdb9)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:55.421483 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:55.431569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-a0dfa7b7-e158-4c24-83fc-1378bc2afeb8 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:55.432757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a0dfa7b7-e158-4c24-83fc-1378bc2afeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:55.440870 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1352/2715] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:55.445120 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfa7b7-e158-4c24-83fc-1378bc2afeb8 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.454876 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dfa7b7-e158-4c24-83fc-1378bc2afeb8 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:55.459529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c0511-e535-4df8-aec8-56a40c1a993a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:55.460166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c0511-e535-4df8-aec8-56a40c1a993a admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.460446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c0511-e535-4df8-aec8-56a40c1a993a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.460719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c0511-e535-4df8-aec8-56a40c1a993a admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.461433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c0511-e535-4df8-aec8-56a40c1a993a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:15:55.461741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c0511-e535-4df8-aec8-56a40c1a993a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:55.463350 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c0511-e535-4df8-aec8-56a40c1a993a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:55.467833 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b9b54f1a5cc4d48a6369a50a611fdb9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.468440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b9b54f1a5cc4d48a6369a50a611fdb9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.475413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1353/2716] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:15:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:15:55.475645 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b9b54f1a5cc4d48a6369a50a611fdb9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.479570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c0511-e535-4df8-aec8-56a40c1a993a admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.486534 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b9b54f1a5cc4d48a6369a50a611fdb9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.487327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b9b54f1a5cc4d48a6369a50a611fdb9'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.489753 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c0511-e535-4df8-aec8-56a40c1a993a admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:55.494666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698b096-be39-4bae-85ec-86b8d0366f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:55.495310 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698b096-be39-4bae-85ec-86b8d0366f85 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.495597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698b096-be39-4bae-85ec-86b8d0366f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.495883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698b096-be39-4bae-85ec-86b8d0366f85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.496566 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698b096-be39-4bae-85ec-86b8d0366f85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:55.498510 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698b096-be39-4bae-85ec-86b8d0366f85 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:55.505613 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1364/2717] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:55 2022] DELETE /identity/v3/users/9b9b54f1a5cc4d48a6369a50a611fdb9 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:55.525707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1354/2718] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:55 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:55.530345 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.531746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:55.598509 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a41f1a-800e-4d10-b354-f71b1883d6b4 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:55.657246 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029944c-e095-4f9d-b930-e50d78c427c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:55.657926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029944c-e095-4f9d-b930-e50d78c427c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.658254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029944c-e095-4f9d-b930-e50d78c427c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.658530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c029944c-e095-4f9d-b930-e50d78c427c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.659240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029944c-e095-4f9d-b930-e50d78c427c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:55.661199 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c029944c-e095-4f9d-b930-e50d78c427c5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:55.682871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1365/2719] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:55 2022] POST /identity/v3/users => generated 284 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:55.687026 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698b096-be39-4bae-85ec-86b8d0366f85 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.697187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698b096-be39-4bae-85ec-86b8d0366f85 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:55.702432 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc33eb-3394-42df-99c6-ef90e1d6dd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:55.703099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc33eb-3394-42df-99c6-ef90e1d6dd48 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.703387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc33eb-3394-42df-99c6-ef90e1d6dd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.703670 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cc33eb-3394-42df-99c6-ef90e1d6dd48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.704431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc33eb-3394-42df-99c6-ef90e1d6dd48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:55.706025 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cc33eb-3394-42df-99c6-ef90e1d6dd48 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:55.717641 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1355/2720] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:55.722086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029944c-e095-4f9d-b930-e50d78c427c5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.734463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c029944c-e095-4f9d-b930-e50d78c427c5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:55.739467 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc41f3d-fa8d-4f9d-9ff4-9ca6377dcaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:55.740188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc41f3d-fa8d-4f9d-9ff4-9ca6377dcaff admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.740493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc41f3d-fa8d-4f9d-9ff4-9ca6377dcaff admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.740770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc41f3d-fa8d-4f9d-9ff4-9ca6377dcaff admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.741558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc41f3d-fa8d-4f9d-9ff4-9ca6377dcaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:55.743178 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc41f3d-fa8d-4f9d-9ff4-9ca6377dcaff admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:55.745327 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:55.758529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1366/2721] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:55.759744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc33eb-3394-42df-99c6-ef90e1d6dd48 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.771734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cc33eb-3394-42df-99c6-ef90e1d6dd48 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:55.778365 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a058eef-852c-4d27-9d85-fab827c11461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:55.780289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058eef-852c-4d27-9d85-fab827c11461 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.780603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058eef-852c-4d27-9d85-fab827c11461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.780906 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a058eef-852c-4d27-9d85-fab827c11461 admin admin] PATH_INFO: `/v3/projects/d110d6a8ae46421da0b790bd2e5ce40a/users/b12f59c2a1694ad0b38e3f3f9b078914/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.796427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc41f3d-fa8d-4f9d-9ff4-9ca6377dcaff admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.798033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a058eef-852c-4d27-9d85-fab827c11461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d110d6a8ae46421da0b790bd2e5ce40a, user_id=b12f59c2a1694ad0b38e3f3f9b078914, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:55.798247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1cc41f3d-fa8d-4f9d-9ff4-9ca6377dcaff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:55.800681 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a058eef-852c-4d27-9d85-fab827c11461 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:55.811730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc41f3d-fa8d-4f9d-9ff4-9ca6377dcaff admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:55.818172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:55.819070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.819356 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.819632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] PATH_INFO: `/v3/projects/d3c1952100194b52b2dab5bc43e9bfb6` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.820877 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1356/2722] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:55 2022] PUT /identity/v3/projects/d110d6a8ae46421da0b790bd2e5ce40a/users/b12f59c2a1694ad0b38e3f3f9b078914/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:55.820998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c1952100194b52b2dab5bc43e9bfb6)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:55.823419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:55.832110 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4425b-0ea8-4a08-af4f-e869396eadde None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.832442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4425b-0ea8-4a08-af4f-e869396eadde None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.832682 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4425b-0ea8-4a08-af4f-e869396eadde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.860976 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-fee4425b-0ea8-4a08-af4f-e869396eadde None None] MFA Rules not processed for user `b12f59c2a1694ad0b38e3f3f9b078914`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:55.862229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fee4425b-0ea8-4a08-af4f-e869396eadde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:55.885279 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c1952100194b52b2dab5bc43e9bfb6', 'request_id': 'req-d44b3258-1c85-428d-bebf-d9d1e661fada'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.891766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c1952100194b52b2dab5bc43e9bfb6', 'request_id': 'req-d44b3258-1c85-428d-bebf-d9d1e661fada'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.900142 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c1952100194b52b2dab5bc43e9bfb6', 'request_id': 'req-d44b3258-1c85-428d-bebf-d9d1e661fada'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.915946 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] The token cache is being invalidate because project d3c1952100194b52b2dab5bc43e9bfb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:55.916378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.917250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d44b3258-1c85-428d-bebf-d9d1e661fada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:55.919056 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1367/2723] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:55 2022] DELETE /identity/v3/projects/d3c1952100194b52b2dab5bc43e9bfb6 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:55.923913 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f92dd-df3d-44ee-bb5f-ffcbc5796701 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:55.924228 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f92dd-df3d-44ee-bb5f-ffcbc5796701 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:55.924632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f92dd-df3d-44ee-bb5f-ffcbc5796701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:55.930839 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1357/2724] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:15:55.943044 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f58f92dd-df3d-44ee-bb5f-ffcbc5796701 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:55.943278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4425b-0ea8-4a08-af4f-e869396eadde None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:55.944228 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f58f92dd-df3d-44ee-bb5f-ffcbc5796701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:55.944671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fee4425b-0ea8-4a08-af4f-e869396eadde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:56.001277 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4425b-0ea8-4a08-af4f-e869396eadde None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.003919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1368/2725] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:15:56.011815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af794bce-ba3d-4b0b-957a-a2d3b5fdee25 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.012107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af794bce-ba3d-4b0b-957a-a2d3b5fdee25 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.012384 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af794bce-ba3d-4b0b-957a-a2d3b5fdee25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.022430 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:56.032758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-af794bce-ba3d-4b0b-957a-a2d3b5fdee25 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:56.034126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-af794bce-ba3d-4b0b-957a-a2d3b5fdee25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:56.042088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1369/2726] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:15:56.046637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af794bce-ba3d-4b0b-957a-a2d3b5fdee25 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.054404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96781f85-7a6e-400d-a6f3-069031ea94d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0dc5b601421a41d1a30a10e86967a920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.055350 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af794bce-ba3d-4b0b-957a-a2d3b5fdee25 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.055492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96781f85-7a6e-400d-a6f3-069031ea94d8 service glance] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.055713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96781f85-7a6e-400d-a6f3-069031ea94d8 service glance] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.056112 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96781f85-7a6e-400d-a6f3-069031ea94d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.060388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7158262-b7aa-4619-b37e-38498d25a1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.061021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158262-b7aa-4619-b37e-38498d25a1a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.061297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158262-b7aa-4619-b37e-38498d25a1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.061566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7158262-b7aa-4619-b37e-38498d25a1a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.062315 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158262-b7aa-4619-b37e-38498d25a1a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:15:56.062618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158262-b7aa-4619-b37e-38498d25a1a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.064186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7158262-b7aa-4619-b37e-38498d25a1a1 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.076092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1370/2727] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:15:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:15:56.080187 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7158262-b7aa-4619-b37e-38498d25a1a1 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.089441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7158262-b7aa-4619-b37e-38498d25a1a1 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.094185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3351113-154d-4974-b56c-0ef1f2d72e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.094843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351113-154d-4974-b56c-0ef1f2d72e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.095129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351113-154d-4974-b56c-0ef1f2d72e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.095419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3351113-154d-4974-b56c-0ef1f2d72e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.096087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3351113-154d-4974-b56c-0ef1f2d72e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.097937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3351113-154d-4974-b56c-0ef1f2d72e5e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.113437 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96781f85-7a6e-400d-a6f3-069031ea94d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.115076 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96781f85-7a6e-400d-a6f3-069031ea94d8 service glance] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.124711 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1371/2728] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:56 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:56.124920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1358/2729] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:15:56.128748 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3351113-154d-4974-b56c-0ef1f2d72e5e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.171458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:56.196309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3351113-154d-4974-b56c-0ef1f2d72e5e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.201990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476afd5-8e17-467c-8369-46e73500204b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.202806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476afd5-8e17-467c-8369-46e73500204b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.203121 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476afd5-8e17-467c-8369-46e73500204b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.203302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c476afd5-8e17-467c-8369-46e73500204b admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.203989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476afd5-8e17-467c-8369-46e73500204b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.205898 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c476afd5-8e17-467c-8369-46e73500204b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.230348 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1372/2730] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:56 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:56.231853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96781f85-7a6e-400d-a6f3-069031ea94d8 service glance] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.243060 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96781f85-7a6e-400d-a6f3-069031ea94d8 service glance] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.248122 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc268d6-2699-4f7e-a19c-9ebef702b9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.248786 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc268d6-2699-4f7e-a19c-9ebef702b9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.249072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc268d6-2699-4f7e-a19c-9ebef702b9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.249346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc268d6-2699-4f7e-a19c-9ebef702b9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.250101 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc268d6-2699-4f7e-a19c-9ebef702b9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.251749 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc268d6-2699-4f7e-a19c-9ebef702b9b6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.262400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1359/2731] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:56.266944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476afd5-8e17-467c-8369-46e73500204b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.277670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c476afd5-8e17-467c-8369-46e73500204b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.282338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97b1113-d670-49b4-9602-86daeb8064ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.283000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b1113-d670-49b4-9602-86daeb8064ec admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.283287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b1113-d670-49b4-9602-86daeb8064ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.283568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97b1113-d670-49b4-9602-86daeb8064ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.284290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b1113-d670-49b4-9602-86daeb8064ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.285878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97b1113-d670-49b4-9602-86daeb8064ec admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.301655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1373/2732] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:56.303975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc268d6-2699-4f7e-a19c-9ebef702b9b6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.314457 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc268d6-2699-4f7e-a19c-9ebef702b9b6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.319446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f86a7-0d5f-407a-9b45-84624665e752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.320319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f86a7-0d5f-407a-9b45-84624665e752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.320603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f86a7-0d5f-407a-9b45-84624665e752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.320874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7f86a7-0d5f-407a-9b45-84624665e752 admin admin] PATH_INFO: `/v3/projects/60b2aa8fce6b40e0b40cff9d472d3fa8/users/d0e34a23a21a40c18d6dc9f14ce6a291/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.335861 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f86a7-0d5f-407a-9b45-84624665e752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b2aa8fce6b40e0b40cff9d472d3fa8, user_id=d0e34a23a21a40c18d6dc9f14ce6a291, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.339331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7f86a7-0d5f-407a-9b45-84624665e752 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.353246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1360/2733] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:56 2022] PUT /identity/v3/projects/60b2aa8fce6b40e0b40cff9d472d3fa8/users/d0e34a23a21a40c18d6dc9f14ce6a291/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:56.365677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04965d30-0902-4fc4-8ae1-afc421485775 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.365965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04965d30-0902-4fc4-8ae1-afc421485775 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.366281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04965d30-0902-4fc4-8ae1-afc421485775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.393561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-04965d30-0902-4fc4-8ae1-afc421485775 None None] MFA Rules not processed for user `d0e34a23a21a40c18d6dc9f14ce6a291`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:56.394795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-04965d30-0902-4fc4-8ae1-afc421485775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:56.462558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1374/2734] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:56.467031 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f86a7-0d5f-407a-9b45-84624665e752 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.536210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7f86a7-0d5f-407a-9b45-84624665e752 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.540975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde1750-11f6-4848-8371-ba5b403d0432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.541759 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde1750-11f6-4848-8371-ba5b403d0432 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.542051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde1750-11f6-4848-8371-ba5b403d0432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.542458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fde1750-11f6-4848-8371-ba5b403d0432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.543148 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde1750-11f6-4848-8371-ba5b403d0432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.545557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fde1750-11f6-4848-8371-ba5b403d0432 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.573529 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1361/2735] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:56 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:56.577442 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04965d30-0902-4fc4-8ae1-afc421485775 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.588603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04965d30-0902-4fc4-8ae1-afc421485775 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.650086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f35eb-9093-4212-83c0-28acde4d404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.650752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f35eb-9093-4212-83c0-28acde4d404c admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.651035 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f35eb-9093-4212-83c0-28acde4d404c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.651309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386f35eb-9093-4212-83c0-28acde4d404c admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.651981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f35eb-9093-4212-83c0-28acde4d404c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.654727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386f35eb-9093-4212-83c0-28acde4d404c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.674247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1375/2736] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:56 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:56.678156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde1750-11f6-4848-8371-ba5b403d0432 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.688370 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fde1750-11f6-4848-8371-ba5b403d0432 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.692973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8497de7-1529-4c32-9c15-8d8940d011fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.693626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8497de7-1529-4c32-9c15-8d8940d011fe admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.693909 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8497de7-1529-4c32-9c15-8d8940d011fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.694219 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8497de7-1529-4c32-9c15-8d8940d011fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.695023 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8497de7-1529-4c32-9c15-8d8940d011fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.696683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8497de7-1529-4c32-9c15-8d8940d011fe admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.709287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1362/2737] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:56.713351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f35eb-9093-4212-83c0-28acde4d404c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.724469 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386f35eb-9093-4212-83c0-28acde4d404c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.729160 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf6701-3ccc-4620-8db6-3b7f0f09e3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.730065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf6701-3ccc-4620-8db6-3b7f0f09e3a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.730365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf6701-3ccc-4620-8db6-3b7f0f09e3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.730685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadf6701-3ccc-4620-8db6-3b7f0f09e3a8 admin admin] PATH_INFO: `/v3/projects/d899aeeadee2443bb6a36ae91469e449/users/ef403cf4d3804f20ab5ffec7f257e623/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.745889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf6701-3ccc-4620-8db6-3b7f0f09e3a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d899aeeadee2443bb6a36ae91469e449, user_id=ef403cf4d3804f20ab5ffec7f257e623, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.748571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadf6701-3ccc-4620-8db6-3b7f0f09e3a8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.761229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1376/2738] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:56 2022] PUT /identity/v3/projects/d899aeeadee2443bb6a36ae91469e449/users/ef403cf4d3804f20ab5ffec7f257e623/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:56.765081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8497de7-1529-4c32-9c15-8d8940d011fe admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.774996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8497de7-1529-4c32-9c15-8d8940d011fe admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.788164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:56.835668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75a14d-a352-438d-8d68-cc378a98bd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.836345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75a14d-a352-438d-8d68-cc378a98bd88 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.836630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75a14d-a352-438d-8d68-cc378a98bd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.836910 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75a14d-a352-438d-8d68-cc378a98bd88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.837630 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75a14d-a352-438d-8d68-cc378a98bd88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.839404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75a14d-a352-438d-8d68-cc378a98bd88 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.849891 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1363/2739] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:15:56.854168 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf6701-3ccc-4620-8db6-3b7f0f09e3a8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:56.865115 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadf6701-3ccc-4620-8db6-3b7f0f09e3a8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:56.869801 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f9a1d-1f06-4a5d-bbef-48e0d224c64c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:56.870729 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f9a1d-1f06-4a5d-bbef-48e0d224c64c admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.871012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f9a1d-1f06-4a5d-bbef-48e0d224c64c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.871288 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f9a1d-1f06-4a5d-bbef-48e0d224c64c admin admin] PATH_INFO: `/v3/projects/d899aeeadee2443bb6a36ae91469e449/users/ef403cf4d3804f20ab5ffec7f257e623/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.873290 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f9a1d-1f06-4a5d-bbef-48e0d224c64c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d899aeeadee2443bb6a36ae91469e449, user_id=ef403cf4d3804f20ab5ffec7f257e623, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:56.876015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f9a1d-1f06-4a5d-bbef-48e0d224c64c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:56.882716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:56.887468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1377/2740] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:56 2022] PUT /identity/v3/projects/d899aeeadee2443bb6a36ae91469e449/users/ef403cf4d3804f20ab5ffec7f257e623/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:15:56.897636 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b8128-2c6d-420b-993f-0cbe1239586f None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:56.897935 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b8128-2c6d-420b-993f-0cbe1239586f None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:56.898248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211b8128-2c6d-420b-993f-0cbe1239586f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:56.926249 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-211b8128-2c6d-420b-993f-0cbe1239586f None None] MFA Rules not processed for user `ef403cf4d3804f20ab5ffec7f257e623`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:56.927518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-211b8128-2c6d-420b-993f-0cbe1239586f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:56.999127 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1364/2741] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:56 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:57.011529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f9a1d-1f06-4a5d-bbef-48e0d224c64c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:57.013235 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-808f9a1d-1f06-4a5d-bbef-48e0d224c64c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:57.070186 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f9a1d-1f06-4a5d-bbef-48e0d224c64c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:57.073752 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab2cbc-1106-4375-aaf0-2ad097ae4d13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:57.074431 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2cbc-1106-4375-aaf0-2ad097ae4d13 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:57.074708 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2cbc-1106-4375-aaf0-2ad097ae4d13 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:57.074984 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab2cbc-1106-4375-aaf0-2ad097ae4d13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:57.084220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab2cbc-1106-4375-aaf0-2ad097ae4d13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:57.085624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab2cbc-1106-4375-aaf0-2ad097ae4d13 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:57.095647 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1378/2742] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:15:57.163800 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b8128-2c6d-420b-993f-0cbe1239586f None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:57.174206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211b8128-2c6d-420b-993f-0cbe1239586f None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:57.177705 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd5d2d-ac10-4051-90f1-d163298c31a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:57.178380 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd5d2d-ac10-4051-90f1-d163298c31a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:57.178663 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd5d2d-ac10-4051-90f1-d163298c31a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:57.178937 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd5d2d-ac10-4051-90f1-d163298c31a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:57.237605 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd5d2d-ac10-4051-90f1-d163298c31a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:57.239093 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd5d2d-ac10-4051-90f1-d163298c31a1 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:57.248416 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1365/2743] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:57.506378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab2cbc-1106-4375-aaf0-2ad097ae4d13 service cinder] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:57.569188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab2cbc-1106-4375-aaf0-2ad097ae4d13 service cinder] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:57.573535 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:57.574413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:57.574691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:57.574962 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] PATH_INFO: `/v3/users/d0e34a23a21a40c18d6dc9f14ce6a291` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:57.576126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0e34a23a21a40c18d6dc9f14ce6a291)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:57.578193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:57.628146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0e34a23a21a40c18d6dc9f14ce6a291'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:57.628872 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0e34a23a21a40c18d6dc9f14ce6a291'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:57.638742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0e34a23a21a40c18d6dc9f14ce6a291'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:57.645088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0e34a23a21a40c18d6dc9f14ce6a291'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:57.645699 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0e34a23a21a40c18d6dc9f14ce6a291'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:57.663387 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1379/2744] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:57 2022] DELETE /identity/v3/users/d0e34a23a21a40c18d6dc9f14ce6a291 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:57.667807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f954-f943-489a-848f-880086c73731 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:57.668300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f954-f943-489a-848f-880086c73731 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:57.668398 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465f954-f943-489a-848f-880086c73731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:57.685639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-6465f954-f943-489a-848f-880086c73731 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:57.686900 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6465f954-f943-489a-848f-880086c73731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:57.719399 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:57.749808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1366/2745] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:57.759661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:57.761080 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:57.787780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:57.823471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0872dc7-be33-466e-9dbe-257a20f138c5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:57.827584 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553bbb-ab78-4057-91dd-98832a12b45e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:57.828234 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553bbb-ab78-4057-91dd-98832a12b45e service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:57.828515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553bbb-ab78-4057-91dd-98832a12b45e service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:57.828792 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05553bbb-ab78-4057-91dd-98832a12b45e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:57.837697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553bbb-ab78-4057-91dd-98832a12b45e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:57.839452 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05553bbb-ab78-4057-91dd-98832a12b45e service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:57.850707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1380/2746] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:58.122963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465f954-f943-489a-848f-880086c73731 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:58.124640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6465f954-f943-489a-848f-880086c73731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:58.141318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465f954-f943-489a-848f-880086c73731 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:58.145960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:58.146857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:58.147171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:58.147452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] PATH_INFO: `/v3/projects/60b2aa8fce6b40e0b40cff9d472d3fa8` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:58.148639 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b2aa8fce6b40e0b40cff9d472d3fa8)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:58.150610 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:58.201902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b2aa8fce6b40e0b40cff9d472d3fa8', 'request_id': 'req-ae6c1803-c5f9-4042-907b-fb01ee1b1792'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.208804 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b2aa8fce6b40e0b40cff9d472d3fa8', 'request_id': 'req-ae6c1803-c5f9-4042-907b-fb01ee1b1792'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.214374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b2aa8fce6b40e0b40cff9d472d3fa8', 'request_id': 'req-ae6c1803-c5f9-4042-907b-fb01ee1b1792'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.225288 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] The token cache is being invalidate because project 60b2aa8fce6b40e0b40cff9d472d3fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:58.225576 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.226237 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.227798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1367/2747] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:58 2022] DELETE /identity/v3/projects/60b2aa8fce6b40e0b40cff9d472d3fa8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:58.231903 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553bbb-ab78-4057-91dd-98832a12b45e service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:58.241362 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05553bbb-ab78-4057-91dd-98832a12b45e service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:58.298603 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:58.299459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:58.299745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:58.300020 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] PATH_INFO: `/v3/users/ef403cf4d3804f20ab5ffec7f257e623` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:58.301179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef403cf4d3804f20ab5ffec7f257e623)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:58.303104 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:58.348546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef403cf4d3804f20ab5ffec7f257e623'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.349225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef403cf4d3804f20ab5ffec7f257e623'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.356100 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef403cf4d3804f20ab5ffec7f257e623'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.363510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef403cf4d3804f20ab5ffec7f257e623'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.364306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-86ce0a3c-1adf-49bc-a63d-c36b6d96ec4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef403cf4d3804f20ab5ffec7f257e623'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.383135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1381/2748] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:58 2022] DELETE /identity/v3/users/ef403cf4d3804f20ab5ffec7f257e623 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:58.663065 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:58.664802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:58.679210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c1803-c5f9-4042-907b-fb01ee1b1792 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:58.740149 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:58.741013 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:58.741298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:58.741578 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] PATH_INFO: `/v3/projects/d899aeeadee2443bb6a36ae91469e449` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:58.742778 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d899aeeadee2443bb6a36ae91469e449)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:58.744700 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:58.792996 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd899aeeadee2443bb6a36ae91469e449', 'request_id': 'req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.802162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd899aeeadee2443bb6a36ae91469e449', 'request_id': 'req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.806262 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:58.808629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd899aeeadee2443bb6a36ae91469e449', 'request_id': 'req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.822845 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] The token cache is being invalidate because project d899aeeadee2443bb6a36ae91469e449 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:15:58.823133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.823777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a0fd5a1b-e8b2-4f80-a45c-1234534f8c28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:58.825952 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1368/2749] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:15:58 2022] DELETE /identity/v3/projects/d899aeeadee2443bb6a36ae91469e449 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:58.831062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e0ef5-e820-4b63-9c57-87e775256ab5 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:58.831582 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e0ef5-e820-4b63-9c57-87e775256ab5 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:58.831705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e0ef5-e820-4b63-9c57-87e775256ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:58.849692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-022e0ef5-e820-4b63-9c57-87e775256ab5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:58.850943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-022e0ef5-e820-4b63-9c57-87e775256ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:58.893696 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:58.913820 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1382/2750] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:58.921593 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9470b-bcca-44f1-820a-d8cf84bc1b27 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:58.921877 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9470b-bcca-44f1-820a-d8cf84bc1b27 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:58.922190 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a9470b-bcca-44f1-820a-d8cf84bc1b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:58.942154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-02a9470b-bcca-44f1-820a-d8cf84bc1b27 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:58.943386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-02a9470b-bcca-44f1-820a-d8cf84bc1b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:58.951921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1369/2751] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:58.956211 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e0ef5-e820-4b63-9c57-87e775256ab5 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:58.965998 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e0ef5-e820-4b63-9c57-87e775256ab5 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:58.970558 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7a14-473a-4978-bfc0-8dc8a521c8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:58.971209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7a14-473a-4978-bfc0-8dc8a521c8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:58.971494 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7a14-473a-4978-bfc0-8dc8a521c8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:58.971770 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e7a14-473a-4978-bfc0-8dc8a521c8d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:58.972493 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7a14-473a-4978-bfc0-8dc8a521c8d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:15:58.972806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7a14-473a-4978-bfc0-8dc8a521c8d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:58.974427 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e7a14-473a-4978-bfc0-8dc8a521c8d5 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:58.986357 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1383/2752] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:15:58.989669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9470b-bcca-44f1-820a-d8cf84bc1b27 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:59.000138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a9470b-bcca-44f1-820a-d8cf84bc1b27 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:59.005955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762395b-38e5-4c8a-a9bb-d4f38a18a48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:59.006674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762395b-38e5-4c8a-a9bb-d4f38a18a48b admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:59.006956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762395b-38e5-4c8a-a9bb-d4f38a18a48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:59.007235 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b762395b-38e5-4c8a-a9bb-d4f38a18a48b admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:59.007961 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762395b-38e5-4c8a-a9bb-d4f38a18a48b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:59.009873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b762395b-38e5-4c8a-a9bb-d4f38a18a48b admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:59.035857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1370/2753] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:15:58 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:15:59.039899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7a14-473a-4978-bfc0-8dc8a521c8d5 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:59.050437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e7a14-473a-4978-bfc0-8dc8a521c8d5 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:59.107225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70e278-c863-4593-b894-2e6fbee35266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:59.107884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70e278-c863-4593-b894-2e6fbee35266 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:59.108164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70e278-c863-4593-b894-2e6fbee35266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:59.108440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a70e278-c863-4593-b894-2e6fbee35266 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:59.109123 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70e278-c863-4593-b894-2e6fbee35266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:59.111065 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a70e278-c863-4593-b894-2e6fbee35266 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:59.135926 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1384/2754] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:15:59 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:15:59.138924 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762395b-38e5-4c8a-a9bb-d4f38a18a48b admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:59.149499 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b762395b-38e5-4c8a-a9bb-d4f38a18a48b admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:59.154505 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e8278-edd5-45ff-a0c4-d667f5b59dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:59.155159 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e8278-edd5-45ff-a0c4-d667f5b59dce admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:59.155444 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e8278-edd5-45ff-a0c4-d667f5b59dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:59.155726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e8278-edd5-45ff-a0c4-d667f5b59dce admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:59.156443 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e8278-edd5-45ff-a0c4-d667f5b59dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:59.158019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e8278-edd5-45ff-a0c4-d667f5b59dce admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:59.168723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1371/2755] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:59.172651 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70e278-c863-4593-b894-2e6fbee35266 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:59.183330 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a70e278-c863-4593-b894-2e6fbee35266 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:59.188616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7e55b-2814-408f-aa18-4c1ebf5d0790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:59.189295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7e55b-2814-408f-aa18-4c1ebf5d0790 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:59.189577 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7e55b-2814-408f-aa18-4c1ebf5d0790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:59.189853 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7e55b-2814-408f-aa18-4c1ebf5d0790 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:59.190608 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7e55b-2814-408f-aa18-4c1ebf5d0790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:59.192257 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7e55b-2814-408f-aa18-4c1ebf5d0790 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:59.203581 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1385/2756] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:15:59.207740 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e8278-edd5-45ff-a0c4-d667f5b59dce admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:59.217777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e8278-edd5-45ff-a0c4-d667f5b59dce admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:59.222745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bb3f9-5872-4321-944e-216b96485934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:59.223612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bb3f9-5872-4321-944e-216b96485934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:59.223896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bb3f9-5872-4321-944e-216b96485934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:59.224175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7bb3f9-5872-4321-944e-216b96485934 admin admin] PATH_INFO: `/v3/projects/9dca2eb007db463eaa32ad8c7879dc4e/users/2d7ad5c6c96d492b96d7cc5e073cfa05/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:59.238316 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bb3f9-5872-4321-944e-216b96485934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dca2eb007db463eaa32ad8c7879dc4e, user_id=2d7ad5c6c96d492b96d7cc5e073cfa05, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:59.240953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7bb3f9-5872-4321-944e-216b96485934 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:59.256065 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1372/2757] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:15:59 2022] PUT /identity/v3/projects/9dca2eb007db463eaa32ad8c7879dc4e/users/2d7ad5c6c96d492b96d7cc5e073cfa05/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:59.268017 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777c61a-e017-4417-a5f3-89ffcc7410f3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:59.268309 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777c61a-e017-4417-a5f3-89ffcc7410f3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:59.268586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f777c61a-e017-4417-a5f3-89ffcc7410f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:59.295252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f777c61a-e017-4417-a5f3-89ffcc7410f3 None None] MFA Rules not processed for user `2d7ad5c6c96d492b96d7cc5e073cfa05`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:59.296476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f777c61a-e017-4417-a5f3-89ffcc7410f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:59.360830 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1386/2758] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:15:59.373779 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bb3f9-5872-4321-944e-216b96485934 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:59.375481 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6c7bb3f9-5872-4321-944e-216b96485934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:59.441272 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7bb3f9-5872-4321-944e-216b96485934 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:59.444862 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883b23f3-4e13-4cff-a112-28a2f36911aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:59.445511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b23f3-4e13-4cff-a112-28a2f36911aa service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:59.445789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b23f3-4e13-4cff-a112-28a2f36911aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:59.446087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883b23f3-4e13-4cff-a112-28a2f36911aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:59.454203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b23f3-4e13-4cff-a112-28a2f36911aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:59.455623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883b23f3-4e13-4cff-a112-28a2f36911aa service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:59.464215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1373/2759] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:15:59.485193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777c61a-e017-4417-a5f3-89ffcc7410f3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:59.527025 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:15:59.549676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f777c61a-e017-4417-a5f3-89ffcc7410f3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:59.554654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:59.555519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:59.555826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:59.556121 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] PATH_INFO: `/v3/users/2d7ad5c6c96d492b96d7cc5e073cfa05` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:59.557588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d7ad5c6c96d492b96d7cc5e073cfa05)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:59.559646 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:59.605505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d7ad5c6c96d492b96d7cc5e073cfa05'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:59.606220 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d7ad5c6c96d492b96d7cc5e073cfa05'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:59.613745 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d7ad5c6c96d492b96d7cc5e073cfa05'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:59.622131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d7ad5c6c96d492b96d7cc5e073cfa05'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:59.622839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d7ad5c6c96d492b96d7cc5e073cfa05'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:15:59.640873 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1387/2760] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:15:59 2022] DELETE /identity/v3/users/2d7ad5c6c96d492b96d7cc5e073cfa05 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:15:59.645333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f1403-8e48-46f7-a7fe-7f1f432dc546 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:59.645626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f1403-8e48-46f7-a7fe-7f1f432dc546 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:59.645908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8f1403-8e48-46f7-a7fe-7f1f432dc546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:59.664954 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-2c8f1403-8e48-46f7-a7fe-7f1f432dc546 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:15:59.666284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2c8f1403-8e48-46f7-a7fe-7f1f432dc546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:59.727097 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1374/2761] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:15:59.737258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:15:59.738779 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:15:59.799297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66273cc8-8f3e-4da4-b34b-2b540828346c admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:15:59.802978 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400438f-6e5d-4cef-ab8a-fb2ce8cd4361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:15:59.803639 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400438f-6e5d-4cef-ab8a-fb2ce8cd4361 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:15:59.803928 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400438f-6e5d-4cef-ab8a-fb2ce8cd4361 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:15:59.804296 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400438f-6e5d-4cef-ab8a-fb2ce8cd4361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:15:59.815319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400438f-6e5d-4cef-ab8a-fb2ce8cd4361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:15:59.816753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f400438f-6e5d-4cef-ab8a-fb2ce8cd4361 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:15:59.826971 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1388/2762] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:16:00.338968 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f1403-8e48-46f7-a7fe-7f1f432dc546 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:00.340418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2c8f1403-8e48-46f7-a7fe-7f1f432dc546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:00.353596 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8f1403-8e48-46f7-a7fe-7f1f432dc546 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:00.363850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:00.364897 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:00.365195 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:00.365542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] PATH_INFO: `/v3/projects/9dca2eb007db463eaa32ad8c7879dc4e` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:00.366683 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dca2eb007db463eaa32ad8c7879dc4e)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:00.368622 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:00.394842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:00.419944 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dca2eb007db463eaa32ad8c7879dc4e', 'request_id': 'req-3d63133d-98d1-48b9-a3e9-1d1b532d209c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:00.427296 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dca2eb007db463eaa32ad8c7879dc4e', 'request_id': 'req-3d63133d-98d1-48b9-a3e9-1d1b532d209c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:00.433032 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dca2eb007db463eaa32ad8c7879dc4e', 'request_id': 'req-3d63133d-98d1-48b9-a3e9-1d1b532d209c'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:00.443742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] The token cache is being invalidate because project 9dca2eb007db463eaa32ad8c7879dc4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:16:00.444030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:00.444660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-3d63133d-98d1-48b9-a3e9-1d1b532d209c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:00.446238 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1375/2763] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:16:00 2022] DELETE /identity/v3/projects/9dca2eb007db463eaa32ad8c7879dc4e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:16:00.451284 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f4892-fa51-4f5d-9488-14ba3721d2aa None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:00.451874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f4892-fa51-4f5d-9488-14ba3721d2aa None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:00.451874 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f4892-fa51-4f5d-9488-14ba3721d2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:00.470578 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ac2f4892-fa51-4f5d-9488-14ba3721d2aa None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:00.471803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ac2f4892-fa51-4f5d-9488-14ba3721d2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:00.532303 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1389/2764] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:16:00.540562 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b8caf-ee89-44a2-9005-8d93894affa5 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:00.540794 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b8caf-ee89-44a2-9005-8d93894affa5 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:00.541072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b8caf-ee89-44a2-9005-8d93894affa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:00.559703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-690b8caf-ee89-44a2-9005-8d93894affa5 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:00.560942 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-690b8caf-ee89-44a2-9005-8d93894affa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:00.569448 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1376/2765] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:00.573600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f4892-fa51-4f5d-9488-14ba3721d2aa None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:00.578496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:00.584879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f4892-fa51-4f5d-9488-14ba3721d2aa None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:00.591504 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2940c15-b1f6-43a8-9ce4-66206e9ca0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:00.592189 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2940c15-b1f6-43a8-9ce4-66206e9ca0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:00.592467 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2940c15-b1f6-43a8-9ce4-66206e9ca0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:00.592746 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2940c15-b1f6-43a8-9ce4-66206e9ca0f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:00.593459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2940c15-b1f6-43a8-9ce4-66206e9ca0f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:16:00.593761 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2940c15-b1f6-43a8-9ce4-66206e9ca0f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:00.595385 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2940c15-b1f6-43a8-9ce4-66206e9ca0f6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:00.607692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1390/2766] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:16:00.611973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b8caf-ee89-44a2-9005-8d93894affa5 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:00.621635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b8caf-ee89-44a2-9005-8d93894affa5 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:00.626352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ee5e0-c936-4dfd-87e3-bfa2def2912d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:00.627036 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ee5e0-c936-4dfd-87e3-bfa2def2912d admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:00.627326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ee5e0-c936-4dfd-87e3-bfa2def2912d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:00.627603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697ee5e0-c936-4dfd-87e3-bfa2def2912d admin admin] PATH_INFO: `/v3/projects` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:00.628320 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ee5e0-c936-4dfd-87e3-bfa2def2912d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:00.630511 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697ee5e0-c936-4dfd-87e3-bfa2def2912d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:00.655587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1377/2767] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:16:00 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:16:00.660029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2940c15-b1f6-43a8-9ce4-66206e9ca0f6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:00.670955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2940c15-b1f6-43a8-9ce4-66206e9ca0f6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:00.730634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84b4e5-5b9f-4931-ba3e-440eb9840c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:00.731329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b4e5-5b9f-4931-ba3e-440eb9840c7f admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:00.732247 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b4e5-5b9f-4931-ba3e-440eb9840c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:00.732536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a84b4e5-5b9f-4931-ba3e-440eb9840c7f admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:00.733249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84b4e5-5b9f-4931-ba3e-440eb9840c7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:00.735245 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a84b4e5-5b9f-4931-ba3e-440eb9840c7f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:00.760074 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1391/2768] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:16:00 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:16:00.761512 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ee5e0-c936-4dfd-87e3-bfa2def2912d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:00.772994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697ee5e0-c936-4dfd-87e3-bfa2def2912d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:00.777797 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea6e90-e452-4d9b-8d4b-bcad0894d929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:00.779013 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea6e90-e452-4d9b-8d4b-bcad0894d929 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:00.779294 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea6e90-e452-4d9b-8d4b-bcad0894d929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:00.779570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ea6e90-e452-4d9b-8d4b-bcad0894d929 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:00.780327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea6e90-e452-4d9b-8d4b-bcad0894d929 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:00.781923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ea6e90-e452-4d9b-8d4b-bcad0894d929 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:00.797126 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1378/2769] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:16:00.801012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84b4e5-5b9f-4931-ba3e-440eb9840c7f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:00.811101 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a84b4e5-5b9f-4931-ba3e-440eb9840c7f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:00.815712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f591cf-925f-443a-bc3c-be47a1c1703c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:00.816386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f591cf-925f-443a-bc3c-be47a1c1703c admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:00.816674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f591cf-925f-443a-bc3c-be47a1c1703c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:00.816952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f591cf-925f-443a-bc3c-be47a1c1703c admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:00.817702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f591cf-925f-443a-bc3c-be47a1c1703c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:00.819322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f591cf-925f-443a-bc3c-be47a1c1703c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:00.829839 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1392/2770] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:16:00.833762 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea6e90-e452-4d9b-8d4b-bcad0894d929 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:00.845114 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ea6e90-e452-4d9b-8d4b-bcad0894d929 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:00.849867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4610dee-3832-42c6-9b6c-6c58a3273ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:00.850798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610dee-3832-42c6-9b6c-6c58a3273ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:00.851082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610dee-3832-42c6-9b6c-6c58a3273ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:00.851376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4610dee-3832-42c6-9b6c-6c58a3273ecf admin admin] PATH_INFO: `/v3/projects/e9a1fcfd6a204a96a2a9a0c318c3a67f/users/8f46f9db65b840458753d8d3ee63a3d1/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:00.866569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4610dee-3832-42c6-9b6c-6c58a3273ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9a1fcfd6a204a96a2a9a0c318c3a67f, user_id=8f46f9db65b840458753d8d3ee63a3d1, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:00.869189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4610dee-3832-42c6-9b6c-6c58a3273ecf admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:00.882500 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1379/2771] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:16:00 2022] PUT /identity/v3/projects/e9a1fcfd6a204a96a2a9a0c318c3a67f/users/8f46f9db65b840458753d8d3ee63a3d1/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:16:00.893498 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4fd06-011d-47ee-b94e-2a5244cf3f97 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:00.893787 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4fd06-011d-47ee-b94e-2a5244cf3f97 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:00.894105 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4fd06-011d-47ee-b94e-2a5244cf3f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:00.922471 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c8b4fd06-011d-47ee-b94e-2a5244cf3f97 None None] MFA Rules not processed for user `8f46f9db65b840458753d8d3ee63a3d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:00.923692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c8b4fd06-011d-47ee-b94e-2a5244cf3f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:00.986863 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1393/2772] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:16:00.997482 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4610dee-3832-42c6-9b6c-6c58a3273ecf admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:00.999203 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f4610dee-3832-42c6-9b6c-6c58a3273ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:01.058106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4610dee-3832-42c6-9b6c-6c58a3273ecf admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:01.061623 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba5a52-3770-4968-bea2-8dc2ecdcbb51 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0dc5b601421a41d1a30a10e86967a920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:01.062301 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba5a52-3770-4968-bea2-8dc2ecdcbb51 service glance] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:01.062586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba5a52-3770-4968-bea2-8dc2ecdcbb51 service glance] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:01.062865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afba5a52-3770-4968-bea2-8dc2ecdcbb51 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:01.070946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba5a52-3770-4968-bea2-8dc2ecdcbb51 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:01.072368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afba5a52-3770-4968-bea2-8dc2ecdcbb51 service glance] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:01.082229 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1380/2773] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:16:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:16:10.984251 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=71,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:11.077534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=68,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:35.433120 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4fd06-011d-47ee-b94e-2a5244cf3f97 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:35.434601 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c8b4fd06-011d-47ee-b94e-2a5244cf3f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:35.493249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4fd06-011d-47ee-b94e-2a5244cf3f97 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:35.497076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc701e6-ca86-442f-9398-d2e6d091ceb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:35.497725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc701e6-ca86-442f-9398-d2e6d091ceb3 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:35.498002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc701e6-ca86-442f-9398-d2e6d091ceb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:35.498307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc701e6-ca86-442f-9398-d2e6d091ceb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:35.500071 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-efc701e6-ca86-442f-9398-d2e6d091ceb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:35.510298 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc701e6-ca86-442f-9398-d2e6d091ceb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:35.511652 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc701e6-ca86-442f-9398-d2e6d091ceb3 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:35.520964 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1394/2774] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:16:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:16:38.478189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba5a52-3770-4968-bea2-8dc2ecdcbb51 service glance] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:38.539367 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afba5a52-3770-4968-bea2-8dc2ecdcbb51 service glance] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:38.543930 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:38.544785 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:38.545073 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:38.545353 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] PATH_INFO: `/v3/users/8f46f9db65b840458753d8d3ee63a3d1` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:38.546568 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f46f9db65b840458753d8d3ee63a3d1)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:38.548466 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:38.592741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f46f9db65b840458753d8d3ee63a3d1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:38.593382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f46f9db65b840458753d8d3ee63a3d1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:38.593981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f46f9db65b840458753d8d3ee63a3d1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:38.605441 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f46f9db65b840458753d8d3ee63a3d1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:38.612563 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f46f9db65b840458753d8d3ee63a3d1'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:38.624702 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1381/2775] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:16:38 2022] DELETE /identity/v3/users/8f46f9db65b840458753d8d3ee63a3d1 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:16:38.629760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114737d-bf7c-465d-b2c8-03517ecb0e11 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:38.630528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114737d-bf7c-465d-b2c8-03517ecb0e11 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:38.630634 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4114737d-bf7c-465d-b2c8-03517ecb0e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:38.648305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4114737d-bf7c-465d-b2c8-03517ecb0e11 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:38.649488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4114737d-bf7c-465d-b2c8-03517ecb0e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:38.706326 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1395/2776] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:38.715885 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:38.717321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:38.774134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9717f8-e748-465d-8b08-cb1a894c7be4 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:38.777798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b6bd4-2e86-4082-bc29-6071c8968a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:38.778484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b6bd4-2e86-4082-bc29-6071c8968a22 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:38.778775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b6bd4-2e86-4082-bc29-6071c8968a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:38.779053 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b6bd4-2e86-4082-bc29-6071c8968a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:38.788170 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b6bd4-2e86-4082-bc29-6071c8968a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:38.789597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b6bd4-2e86-4082-bc29-6071c8968a22 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:38.799581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1382/2777] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:16:39.154698 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114737d-bf7c-465d-b2c8-03517ecb0e11 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:39.156446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4114737d-bf7c-465d-b2c8-03517ecb0e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:39.171302 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4114737d-bf7c-465d-b2c8-03517ecb0e11 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:39.176959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:39.177828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.178140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.178461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] PATH_INFO: `/v3/projects/e9a1fcfd6a204a96a2a9a0c318c3a67f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.179850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9a1fcfd6a204a96a2a9a0c318c3a67f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:39.182126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:39.244660 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9a1fcfd6a204a96a2a9a0c318c3a67f', 'request_id': 'req-68de205e-8b55-4bc7-97f3-bb630da4084b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:39.250174 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9a1fcfd6a204a96a2a9a0c318c3a67f', 'request_id': 'req-68de205e-8b55-4bc7-97f3-bb630da4084b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:39.258769 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9a1fcfd6a204a96a2a9a0c318c3a67f', 'request_id': 'req-68de205e-8b55-4bc7-97f3-bb630da4084b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:39.270749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] The token cache is being invalidate because project e9a1fcfd6a204a96a2a9a0c318c3a67f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:16:39.271039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:39.271680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-68de205e-8b55-4bc7-97f3-bb630da4084b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:39.273214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1396/2778] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:16:39 2022] DELETE /identity/v3/projects/e9a1fcfd6a204a96a2a9a0c318c3a67f => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:16:39.278178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d47142-cbfd-4865-b338-f583a8e8c6ea None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.278470 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d47142-cbfd-4865-b338-f583a8e8c6ea None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.278748 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d47142-cbfd-4865-b338-f583a8e8c6ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.295916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-c7d47142-cbfd-4865-b338-f583a8e8c6ea None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:39.297103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c7d47142-cbfd-4865-b338-f583a8e8c6ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:39.315780 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:39.358513 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1383/2779] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:39.366916 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59c445-d5a3-4378-8bbf-8d1014f81dfe None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.367209 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59c445-d5a3-4378-8bbf-8d1014f81dfe None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.367492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59c445-d5a3-4378-8bbf-8d1014f81dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.386064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-9c59c445-d5a3-4378-8bbf-8d1014f81dfe None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:39.387266 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9c59c445-d5a3-4378-8bbf-8d1014f81dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:39.395046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1397/2780] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:39.399635 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d47142-cbfd-4865-b338-f583a8e8c6ea None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:39.409802 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d47142-cbfd-4865-b338-f583a8e8c6ea None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:39.414285 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171da304-daf8-44a8-984d-de4a929c3af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:39.414928 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171da304-daf8-44a8-984d-de4a929c3af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.415213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171da304-daf8-44a8-984d-de4a929c3af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.415484 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171da304-daf8-44a8-984d-de4a929c3af2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.416232 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171da304-daf8-44a8-984d-de4a929c3af2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:16:39.416538 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171da304-daf8-44a8-984d-de4a929c3af2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:39.418140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171da304-daf8-44a8-984d-de4a929c3af2 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:39.430468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1384/2781] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:16:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:16:39.434599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59c445-d5a3-4378-8bbf-8d1014f81dfe None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:39.443870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c59c445-d5a3-4378-8bbf-8d1014f81dfe None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:39.448417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2dd57-2fc1-4122-87b6-09e05cd02d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:39.449053 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2dd57-2fc1-4122-87b6-09e05cd02d6e admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.449364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2dd57-2fc1-4122-87b6-09e05cd02d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.449636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e2dd57-2fc1-4122-87b6-09e05cd02d6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.450510 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2dd57-2fc1-4122-87b6-09e05cd02d6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:39.452458 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e2dd57-2fc1-4122-87b6-09e05cd02d6e admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:39.472022 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:39.480961 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1398/2782] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:16:39 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:16:39.485693 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171da304-daf8-44a8-984d-de4a929c3af2 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:39.499087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171da304-daf8-44a8-984d-de4a929c3af2 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:39.559178 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b3e56-8016-47ea-abec-8f6d1176219c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:39.559853 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b3e56-8016-47ea-abec-8f6d1176219c admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.560136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b3e56-8016-47ea-abec-8f6d1176219c admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.560413 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b3e56-8016-47ea-abec-8f6d1176219c admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.561113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b3e56-8016-47ea-abec-8f6d1176219c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:39.563211 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b3e56-8016-47ea-abec-8f6d1176219c admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:39.587854 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1385/2783] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:16:39 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:16:39.589358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2dd57-2fc1-4122-87b6-09e05cd02d6e admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:39.600438 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e2dd57-2fc1-4122-87b6-09e05cd02d6e admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:39.605054 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fc5b5-947a-479d-97f7-9fa86dfb18ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:39.605717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fc5b5-947a-479d-97f7-9fa86dfb18ef admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.605997 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fc5b5-947a-479d-97f7-9fa86dfb18ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.606307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15fc5b5-947a-479d-97f7-9fa86dfb18ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.607023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fc5b5-947a-479d-97f7-9fa86dfb18ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:39.608615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15fc5b5-947a-479d-97f7-9fa86dfb18ef admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:39.620715 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1399/2784] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:16:39.624805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b3e56-8016-47ea-abec-8f6d1176219c admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:39.635395 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b3e56-8016-47ea-abec-8f6d1176219c admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:39.640132 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911a06b-08c3-495f-9b75-d5a54968783d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:39.640805 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911a06b-08c3-495f-9b75-d5a54968783d admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.641125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911a06b-08c3-495f-9b75-d5a54968783d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.641422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911a06b-08c3-495f-9b75-d5a54968783d admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.642189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911a06b-08c3-495f-9b75-d5a54968783d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:39.643789 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911a06b-08c3-495f-9b75-d5a54968783d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:39.654250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1386/2785] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:16:39.659242 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fc5b5-947a-479d-97f7-9fa86dfb18ef admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:39.670316 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15fc5b5-947a-479d-97f7-9fa86dfb18ef admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:39.676721 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d83df-d247-4c5a-9dcf-7307b1e7acd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:39.677605 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d83df-d247-4c5a-9dcf-7307b1e7acd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.677885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d83df-d247-4c5a-9dcf-7307b1e7acd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.678197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d83df-d247-4c5a-9dcf-7307b1e7acd2 admin admin] PATH_INFO: `/v3/projects/ebd394696c824dcd98d3a1304bf4a48c/users/4496b0c516af457a8792c2a243e74668/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.694273 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d83df-d247-4c5a-9dcf-7307b1e7acd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebd394696c824dcd98d3a1304bf4a48c, user_id=4496b0c516af457a8792c2a243e74668, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:39.697172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d83df-d247-4c5a-9dcf-7307b1e7acd2 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:39.711909 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1400/2786] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:16:39 2022] PUT /identity/v3/projects/ebd394696c824dcd98d3a1304bf4a48c/users/4496b0c516af457a8792c2a243e74668/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:16:39.723309 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e785d9-3b87-4e39-a9ad-9c1031059b04 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.723555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e785d9-3b87-4e39-a9ad-9c1031059b04 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.723813 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e785d9-3b87-4e39-a9ad-9c1031059b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.753938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-d0e785d9-3b87-4e39-a9ad-9c1031059b04 None None] MFA Rules not processed for user `4496b0c516af457a8792c2a243e74668`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:39.755717 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-d0e785d9-3b87-4e39-a9ad-9c1031059b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:39.822612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1387/2787] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:39.834128 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d83df-d247-4c5a-9dcf-7307b1e7acd2 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:39.835884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-aa5d83df-d247-4c5a-9dcf-7307b1e7acd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:39.898328 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d83df-d247-4c5a-9dcf-7307b1e7acd2 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:39.902036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550c964-0b6b-4c26-aa84-dff4d671fb95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:39.902702 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550c964-0b6b-4c26-aa84-dff4d671fb95 service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:39.902992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550c964-0b6b-4c26-aa84-dff4d671fb95 service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:39.903269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550c964-0b6b-4c26-aa84-dff4d671fb95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:39.912675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550c964-0b6b-4c26-aa84-dff4d671fb95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:39.914140 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550c964-0b6b-4c26-aa84-dff4d671fb95 service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:39.923408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1401/2788] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:16:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:16:49.819599 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:49.919446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=42,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:52.418660 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e785d9-3b87-4e39-a9ad-9c1031059b04 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:52.472117 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e785d9-3b87-4e39-a9ad-9c1031059b04 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:52.476492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:52.477345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:52.477624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:52.477898 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] PATH_INFO: `/v3/users/4496b0c516af457a8792c2a243e74668` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:52.479089 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4496b0c516af457a8792c2a243e74668)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:52.480981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:52.525735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4496b0c516af457a8792c2a243e74668'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:52.526405 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4496b0c516af457a8792c2a243e74668'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:52.527008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4496b0c516af457a8792c2a243e74668'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:52.539181 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4496b0c516af457a8792c2a243e74668'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:52.546387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4496b0c516af457a8792c2a243e74668'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:52.558402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1388/2789] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:16:52 2022] DELETE /identity/v3/users/4496b0c516af457a8792c2a243e74668 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:16:52.562773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea4f42-7feb-4bbf-aab5-b15db20b57af None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:52.563281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea4f42-7feb-4bbf-aab5-b15db20b57af None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:52.563391 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea4f42-7feb-4bbf-aab5-b15db20b57af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:52.580569 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-c9ea4f42-7feb-4bbf-aab5-b15db20b57af None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:52.581771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c9ea4f42-7feb-4bbf-aab5-b15db20b57af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:52.639366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1402/2790] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:52.649246 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:52.650699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:52.706179 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96269d8-d8f3-4b7c-9efe-353c876dfb1f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:52.710278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3eecc-aaf5-43b7-99bd-786cef39b894 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:52.710919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3eecc-aaf5-43b7-99bd-786cef39b894 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:52.711204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3eecc-aaf5-43b7-99bd-786cef39b894 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:52.711483 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3eecc-aaf5-43b7-99bd-786cef39b894 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:52.723289 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3eecc-aaf5-43b7-99bd-786cef39b894 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:52.725204 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3eecc-aaf5-43b7-99bd-786cef39b894 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:52.734742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1389/2791] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:16:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:16:52.994607 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea4f42-7feb-4bbf-aab5-b15db20b57af None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:52.996098 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c9ea4f42-7feb-4bbf-aab5-b15db20b57af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:53.010992 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea4f42-7feb-4bbf-aab5-b15db20b57af None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:53.015599 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:53.017113 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.018068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.018366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] PATH_INFO: `/v3/projects/ebd394696c824dcd98d3a1304bf4a48c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.019767 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebd394696c824dcd98d3a1304bf4a48c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:53.021723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:53.066959 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebd394696c824dcd98d3a1304bf4a48c', 'request_id': 'req-fc52d0d9-c0be-4244-8538-dc88e073ef5a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:53.072617 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebd394696c824dcd98d3a1304bf4a48c', 'request_id': 'req-fc52d0d9-c0be-4244-8538-dc88e073ef5a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:53.080012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebd394696c824dcd98d3a1304bf4a48c', 'request_id': 'req-fc52d0d9-c0be-4244-8538-dc88e073ef5a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:53.091053 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] The token cache is being invalidate because project ebd394696c824dcd98d3a1304bf4a48c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:16:53.091337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:53.092067 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-fc52d0d9-c0be-4244-8538-dc88e073ef5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:53.093697 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1403/2792] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:16:52 2022] DELETE /identity/v3/projects/ebd394696c824dcd98d3a1304bf4a48c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:16:53.098586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc3fa-ff6b-4972-b465-e446e55caeee None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.098883 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc3fa-ff6b-4972-b465-e446e55caeee None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.099189 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fdc3fa-ff6b-4972-b465-e446e55caeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.116653 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-37fdc3fa-ff6b-4972-b465-e446e55caeee None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:53.117868 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-37fdc3fa-ff6b-4972-b465-e446e55caeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:53.135763 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:53.181172 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1390/2793] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:53.190372 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e6a9a-a8a9-4731-b6b6-1661f14950b0 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.190678 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e6a9a-a8a9-4731-b6b6-1661f14950b0 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.190938 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e6a9a-a8a9-4731-b6b6-1661f14950b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.209548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-354e6a9a-a8a9-4731-b6b6-1661f14950b0 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:53.210782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-354e6a9a-a8a9-4731-b6b6-1661f14950b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:53.218579 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1404/2794] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:16:53.222735 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc3fa-ff6b-4972-b465-e446e55caeee None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:53.232573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fdc3fa-ff6b-4972-b465-e446e55caeee None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:53.237672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f93cd-2090-4acb-9214-617d95d1a757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:53.238364 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f93cd-2090-4acb-9214-617d95d1a757 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.238648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f93cd-2090-4acb-9214-617d95d1a757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.238920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f93cd-2090-4acb-9214-617d95d1a757 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.239640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f93cd-2090-4acb-9214-617d95d1a757 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:16:53.239945 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f93cd-2090-4acb-9214-617d95d1a757 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:53.241515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f93cd-2090-4acb-9214-617d95d1a757 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:53.253895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1391/2795] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:16:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:16:53.258412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e6a9a-a8a9-4731-b6b6-1661f14950b0 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:53.267615 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e6a9a-a8a9-4731-b6b6-1661f14950b0 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:53.272522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83b722-d392-4208-87bc-b540748cf953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:53.273253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83b722-d392-4208-87bc-b540748cf953 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.273528 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83b722-d392-4208-87bc-b540748cf953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.273800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e83b722-d392-4208-87bc-b540748cf953 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.274521 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83b722-d392-4208-87bc-b540748cf953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:53.276389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e83b722-d392-4208-87bc-b540748cf953 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:53.301539 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1405/2796] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:16:53 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:16:53.305781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f93cd-2090-4acb-9214-617d95d1a757 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:53.316620 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f93cd-2090-4acb-9214-617d95d1a757 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:53.372158 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e639c3e-5f10-4e01-80e4-be157f1582a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:53.372821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e639c3e-5f10-4e01-80e4-be157f1582a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.373106 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e639c3e-5f10-4e01-80e4-be157f1582a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.373382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e639c3e-5f10-4e01-80e4-be157f1582a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.374087 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e639c3e-5f10-4e01-80e4-be157f1582a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:53.376710 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e639c3e-5f10-4e01-80e4-be157f1582a8 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:53.395894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1392/2797] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:16:53 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:16:53.400084 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83b722-d392-4208-87bc-b540748cf953 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:53.410347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e83b722-d392-4208-87bc-b540748cf953 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:53.414883 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236945e-f6dc-4acd-91f3-6c7b62dfad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:53.415537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236945e-f6dc-4acd-91f3-6c7b62dfad98 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.415822 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236945e-f6dc-4acd-91f3-6c7b62dfad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.416099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236945e-f6dc-4acd-91f3-6c7b62dfad98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.416817 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2236945e-f6dc-4acd-91f3-6c7b62dfad98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:53.418434 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2236945e-f6dc-4acd-91f3-6c7b62dfad98 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:53.429808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1406/2798] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:16:53.434164 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e639c3e-5f10-4e01-80e4-be157f1582a8 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:53.444896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e639c3e-5f10-4e01-80e4-be157f1582a8 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:53.449704 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da9aa94-c338-457a-9bc9-fd976ece3534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:53.450386 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9aa94-c338-457a-9bc9-fd976ece3534 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.450669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9aa94-c338-457a-9bc9-fd976ece3534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.450941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da9aa94-c338-457a-9bc9-fd976ece3534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.451722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9aa94-c338-457a-9bc9-fd976ece3534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:53.453208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da9aa94-c338-457a-9bc9-fd976ece3534 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:53.465051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1393/2799] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:16:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:16:53.469270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236945e-f6dc-4acd-91f3-6c7b62dfad98 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:53.480206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236945e-f6dc-4acd-91f3-6c7b62dfad98 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:53.485965 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb6789-4b99-4a79-a7c3-c3e141d33323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:53.486910 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb6789-4b99-4a79-a7c3-c3e141d33323 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.487332 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb6789-4b99-4a79-a7c3-c3e141d33323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.487619 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb6789-4b99-4a79-a7c3-c3e141d33323 admin admin] PATH_INFO: `/v3/projects/182e5a14c6924d489d7f50ac802e18dd/users/0450840afbc14330b4f89a35a1327881/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.504397 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb6789-4b99-4a79-a7c3-c3e141d33323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=182e5a14c6924d489d7f50ac802e18dd, user_id=0450840afbc14330b4f89a35a1327881, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:53.507319 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb6789-4b99-4a79-a7c3-c3e141d33323 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:53.521485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1407/2800] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:16:53 2022] PUT /identity/v3/projects/182e5a14c6924d489d7f50ac802e18dd/users/0450840afbc14330b4f89a35a1327881/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:16:53.533000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656a390-bbc3-4613-be3e-6c4e25f1134d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.533297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656a390-bbc3-4613-be3e-6c4e25f1134d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.533577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656a390-bbc3-4613-be3e-6c4e25f1134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.564503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b656a390-bbc3-4613-be3e-6c4e25f1134d None None] MFA Rules not processed for user `0450840afbc14330b4f89a35a1327881`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:53.565736 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b656a390-bbc3-4613-be3e-6c4e25f1134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:53.627916 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1394/2801] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:16:53.639258 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb6789-4b99-4a79-a7c3-c3e141d33323 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:53.640950 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8abb6789-4b99-4a79-a7c3-c3e141d33323 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:53.663167 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:53.701747 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb6789-4b99-4a79-a7c3-c3e141d33323 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:53.705513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbe71c3-b700-4f24-bbec-1f51085dee3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:53.706184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe71c3-b700-4f24-bbec-1f51085dee3a service cinder] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.706513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe71c3-b700-4f24-bbec-1f51085dee3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.706758 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbe71c3-b700-4f24-bbec-1f51085dee3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.715276 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe71c3-b700-4f24-bbec-1f51085dee3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:53.716734 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbe71c3-b700-4f24-bbec-1f51085dee3a service cinder] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:53.726307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1408/2802] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:16:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:16:53.750429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656a390-bbc3-4613-be3e-6c4e25f1134d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:53.770461 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:53.815501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656a390-bbc3-4613-be3e-6c4e25f1134d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:53.820083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:53.820941 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.821221 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.821490 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] PATH_INFO: `/v3/users/0450840afbc14330b4f89a35a1327881` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.822692 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0450840afbc14330b4f89a35a1327881)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:53.824589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:53.869701 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0450840afbc14330b4f89a35a1327881'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:53.870417 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0450840afbc14330b4f89a35a1327881'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:53.871061 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0450840afbc14330b4f89a35a1327881'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:53.883367 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0450840afbc14330b4f89a35a1327881'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:53.890293 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0450840afbc14330b4f89a35a1327881'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:53.902083 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1395/2803] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:16:53 2022] DELETE /identity/v3/users/0450840afbc14330b4f89a35a1327881 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:16:53.906620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18972721-f026-433e-825f-cb7161b8c9f4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:53.907423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18972721-f026-433e-825f-cb7161b8c9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:53.907705 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18972721-f026-433e-825f-cb7161b8c9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:53.925063 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-18972721-f026-433e-825f-cb7161b8c9f4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:53.926304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-18972721-f026-433e-825f-cb7161b8c9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:53.984519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1409/2804] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:53.994258 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:53.995654 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:54.052143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339ae2dc-d67f-4fc6-90f3-0352882f37a4 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:54.056010 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e4f47-0c31-4bee-896a-8c86472ecae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:54.056666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e4f47-0c31-4bee-896a-8c86472ecae1 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.056953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e4f47-0c31-4bee-896a-8c86472ecae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.057287 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e4f47-0c31-4bee-896a-8c86472ecae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.066638 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e4f47-0c31-4bee-896a-8c86472ecae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:54.068070 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec1e4f47-0c31-4bee-896a-8c86472ecae1 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:54.077458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1396/2805] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:16:54.372062 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18972721-f026-433e-825f-cb7161b8c9f4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:54.373625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-18972721-f026-433e-825f-cb7161b8c9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:54.388009 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18972721-f026-433e-825f-cb7161b8c9f4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:54.393914 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:54.394849 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.395131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.395404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] PATH_INFO: `/v3/projects/182e5a14c6924d489d7f50ac802e18dd` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.396848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=182e5a14c6924d489d7f50ac802e18dd)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:54.398831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:54.457850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182e5a14c6924d489d7f50ac802e18dd', 'request_id': 'req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:54.465010 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182e5a14c6924d489d7f50ac802e18dd', 'request_id': 'req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:54.474879 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182e5a14c6924d489d7f50ac802e18dd', 'request_id': 'req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:54.487797 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] The token cache is being invalidate because project 182e5a14c6924d489d7f50ac802e18dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:16:54.488043 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:54.488960 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-a9e41e32-02cc-4fcb-a1c1-28956bc1e367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:54.490563 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1410/2806] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:16:54 2022] DELETE /identity/v3/projects/182e5a14c6924d489d7f50ac802e18dd => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:16:54.495373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269f986-d372-4e32-bad1-26456034b566 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.495855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269f986-d372-4e32-bad1-26456034b566 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.495973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269f986-d372-4e32-bad1-26456034b566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.514426 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-0269f986-d372-4e32-bad1-26456034b566 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:54.515651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0269f986-d372-4e32-bad1-26456034b566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:54.549699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:54.583919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1397/2807] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:54.591618 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5effa4-1fda-4553-b617-3dff86e65d7c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.592085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5effa4-1fda-4553-b617-3dff86e65d7c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.592395 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5effa4-1fda-4553-b617-3dff86e65d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.601578 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269f986-d372-4e32-bad1-26456034b566 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:54.603029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0269f986-d372-4e32-bad1-26456034b566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:54.610870 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8e5effa4-1fda-4553-b617-3dff86e65d7c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:54.612107 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8e5effa4-1fda-4553-b617-3dff86e65d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:54.614980 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269f986-d372-4e32-bad1-26456034b566 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:54.619742 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:54.620396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1411/2808] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:54.620666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.620956 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.621227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] PATH_INFO: `/v3/users/b12f59c2a1694ad0b38e3f3f9b078914` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.622542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b12f59c2a1694ad0b38e3f3f9b078914)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:54.624102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5effa4-1fda-4553-b617-3dff86e65d7c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:54.624478 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:54.634015 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5effa4-1fda-4553-b617-3dff86e65d7c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:54.638780 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328e94e-938f-4775-90e0-557e375379db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:54.639450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328e94e-938f-4775-90e0-557e375379db admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.640036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328e94e-938f-4775-90e0-557e375379db admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.640036 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328e94e-938f-4775-90e0-557e375379db admin admin] PATH_INFO: `/v3/domains` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.640815 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328e94e-938f-4775-90e0-557e375379db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80102) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:16:54.641743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328e94e-938f-4775-90e0-557e375379db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:54.643354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328e94e-938f-4775-90e0-557e375379db admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:54.659929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1412/2809] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:16:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 26 03:16:54.660906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328e94e-938f-4775-90e0-557e375379db admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:54.670484 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328e94e-938f-4775-90e0-557e375379db admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:54.672829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b12f59c2a1694ad0b38e3f3f9b078914'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:54.673743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b12f59c2a1694ad0b38e3f3f9b078914'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:54.674402 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b12f59c2a1694ad0b38e3f3f9b078914'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:54.676146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eccca3-2582-4d94-b308-983d3290b9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:54.676785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eccca3-2582-4d94-b308-983d3290b9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.677066 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eccca3-2582-4d94-b308-983d3290b9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.677438 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9eccca3-2582-4d94-b308-983d3290b9c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.678146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eccca3-2582-4d94-b308-983d3290b9c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:54.680092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9eccca3-2582-4d94-b308-983d3290b9c3 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:54.687475 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b12f59c2a1694ad0b38e3f3f9b078914'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:54.696355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-69333b63-7b59-4881-9a3c-42b2e62e913a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b12f59c2a1694ad0b38e3f3f9b078914'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:54.708937 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1413/2810] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:16:54 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:16:54.711637 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1398/2811] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:16:54 2022] DELETE /identity/v3/users/b12f59c2a1694ad0b38e3f3f9b078914 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:16:54.715225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eccca3-2582-4d94-b308-983d3290b9c3 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:54.715934 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed723e98-650c-4e74-a30e-69071d183807 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.716433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed723e98-650c-4e74-a30e-69071d183807 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.716717 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed723e98-650c-4e74-a30e-69071d183807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.717001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f9eccca3-2582-4d94-b308-983d3290b9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:54.723623 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:54.736917 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-ed723e98-650c-4e74-a30e-69071d183807 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:54.738144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ed723e98-650c-4e74-a30e-69071d183807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:54.788088 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9eccca3-2582-4d94-b308-983d3290b9c3 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:54.802551 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1399/2812] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:54.811698 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed723e98-650c-4e74-a30e-69071d183807 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:54.813666 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ed723e98-650c-4e74-a30e-69071d183807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:54.844915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d56b4b-6708-456d-b2ce-c75e375206b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:54.845597 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d56b4b-6708-456d-b2ce-c75e375206b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.845882 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d56b4b-6708-456d-b2ce-c75e375206b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.846193 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d56b4b-6708-456d-b2ce-c75e375206b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.846887 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d56b4b-6708-456d-b2ce-c75e375206b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:54.848805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d56b4b-6708-456d-b2ce-c75e375206b6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:54.873993 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1414/2813] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:16:54 2022] POST /identity/v3/users => generated 288 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:16:54.874771 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d56b4b-6708-456d-b2ce-c75e375206b6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:54.883611 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed723e98-650c-4e74-a30e-69071d183807 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:54.885365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d56b4b-6708-456d-b2ce-c75e375206b6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:54.887240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d6f46-a4e1-4df0-a0ad-35b4794cb7d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:54.887890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d6f46-a4e1-4df0-a0ad-35b4794cb7d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.888174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d6f46-a4e1-4df0-a0ad-35b4794cb7d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.888455 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7d6f46-a4e1-4df0-a0ad-35b4794cb7d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.889983 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af286744-54be-4761-9311-dfeeeb39e810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:54.890649 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af286744-54be-4761-9311-dfeeeb39e810 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.890930 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af286744-54be-4761-9311-dfeeeb39e810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.891237 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af286744-54be-4761-9311-dfeeeb39e810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.891980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af286744-54be-4761-9311-dfeeeb39e810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:54.893551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af286744-54be-4761-9311-dfeeeb39e810 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:54.896959 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:54.899830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d6f46-a4e1-4df0-a0ad-35b4794cb7d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:54.901554 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7d6f46-a4e1-4df0-a0ad-35b4794cb7d8 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:54.906783 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1415/2814] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:16:54.907804 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af286744-54be-4761-9311-dfeeeb39e810 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:54.912955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1400/2815] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:16:54.917394 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af286744-54be-4761-9311-dfeeeb39e810 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:54.921987 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d77432-b347-4241-8b95-f79de36ffdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:54.923208 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d77432-b347-4241-8b95-f79de36ffdbb admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.923488 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d77432-b347-4241-8b95-f79de36ffdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.923765 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d77432-b347-4241-8b95-f79de36ffdbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.924482 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d77432-b347-4241-8b95-f79de36ffdbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:54.926068 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d77432-b347-4241-8b95-f79de36ffdbb admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:54.938552 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1416/2816] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:16:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:16:54.942421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d6f46-a4e1-4df0-a0ad-35b4794cb7d8 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:54.952079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7d6f46-a4e1-4df0-a0ad-35b4794cb7d8 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:54.956923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4c213-d1ee-45c7-9f90-1c3d43c70afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:54.957803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4c213-d1ee-45c7-9f90-1c3d43c70afb admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:54.958130 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4c213-d1ee-45c7-9f90-1c3d43c70afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:54.958398 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4c213-d1ee-45c7-9f90-1c3d43c70afb admin admin] PATH_INFO: `/v3/projects/1cd719320fd44f72a68396014351b954/users/64150f0f08454ac3801e75bad70980ae/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:54.974607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4c213-d1ee-45c7-9f90-1c3d43c70afb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cd719320fd44f72a68396014351b954, user_id=64150f0f08454ac3801e75bad70980ae, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:54.979669 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4c213-d1ee-45c7-9f90-1c3d43c70afb admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:54.996205 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1401/2817] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:16:54 2022] PUT /identity/v3/projects/1cd719320fd44f72a68396014351b954/users/64150f0f08454ac3801e75bad70980ae/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:16:55.008681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29e162-624a-49d7-ab3b-a13a378c1a54 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.008970 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29e162-624a-49d7-ab3b-a13a378c1a54 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.009248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b29e162-624a-49d7-ab3b-a13a378c1a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.035404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1b29e162-624a-49d7-ab3b-a13a378c1a54 None None] MFA Rules not processed for user `64150f0f08454ac3801e75bad70980ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:55.036633 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1b29e162-624a-49d7-ab3b-a13a378c1a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:55.113090 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1417/2818] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:55 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:55.123922 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4c213-d1ee-45c7-9f90-1c3d43c70afb admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:55.125656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-1ca4c213-d1ee-45c7-9f90-1c3d43c70afb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:55.182955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4c213-d1ee-45c7-9f90-1c3d43c70afb admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:55.187278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2d94b4-0d95-499c-b3cd-ea5885b22789 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:55.187981 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d94b4-0d95-499c-b3cd-ea5885b22789 service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.188374 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d94b4-0d95-499c-b3cd-ea5885b22789 service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.188770 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2d94b4-0d95-499c-b3cd-ea5885b22789 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.199042 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d94b4-0d95-499c-b3cd-ea5885b22789 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:55.200757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2d94b4-0d95-499c-b3cd-ea5885b22789 service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:55.207766 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29e162-624a-49d7-ab3b-a13a378c1a54 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:55.209516 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1b29e162-624a-49d7-ab3b-a13a378c1a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:55.210686 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1402/2819] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:16:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:16:55.222459 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:55.285496 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b29e162-624a-49d7-ab3b-a13a378c1a54 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:55.290153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:55.291016 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.291322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.291598 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] PATH_INFO: `/v3/projects/d110d6a8ae46421da0b790bd2e5ce40a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.292775 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d110d6a8ae46421da0b790bd2e5ce40a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:55.294704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:55.344943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd110d6a8ae46421da0b790bd2e5ce40a', 'request_id': 'req-b3637abf-8768-4b07-b77c-2ebe7012de17'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:55.351218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd110d6a8ae46421da0b790bd2e5ce40a', 'request_id': 'req-b3637abf-8768-4b07-b77c-2ebe7012de17'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:55.359460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd110d6a8ae46421da0b790bd2e5ce40a', 'request_id': 'req-b3637abf-8768-4b07-b77c-2ebe7012de17'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:55.371676 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] The token cache is being invalidate because project d110d6a8ae46421da0b790bd2e5ce40a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:16:55.371967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:55.372663 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b3637abf-8768-4b07-b77c-2ebe7012de17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:16:55.374505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1418/2820] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:16:55 2022] DELETE /identity/v3/projects/d110d6a8ae46421da0b790bd2e5ce40a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:16:55.379527 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ac785-0f42-4b93-9c42-d94fb69c98e1 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.379808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ac785-0f42-4b93-9c42-d94fb69c98e1 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.380831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ac785-0f42-4b93-9c42-d94fb69c98e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.398942 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-544ac785-0f42-4b93-9c42-d94fb69c98e1 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:55.400162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-544ac785-0f42-4b93-9c42-d94fb69c98e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:55.464161 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1403/2821] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:16:55.472425 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65477d-4d7d-4986-8a3a-8b070e448bef None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.472723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65477d-4d7d-4986-8a3a-8b070e448bef None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.473033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d65477d-4d7d-4986-8a3a-8b070e448bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.493262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-4d65477d-4d7d-4986-8a3a-8b070e448bef None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:55.494475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-4d65477d-4d7d-4986-8a3a-8b070e448bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:55.502832 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1419/2822] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:55.507442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ac785-0f42-4b93-9c42-d94fb69c98e1 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:55.518098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ac785-0f42-4b93-9c42-d94fb69c98e1 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:55.522842 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdb1a7-9487-40e3-ae96-a5fd41de66e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:55.523508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdb1a7-9487-40e3-ae96-a5fd41de66e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.523795 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdb1a7-9487-40e3-ae96-a5fd41de66e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.524072 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cdb1a7-9487-40e3-ae96-a5fd41de66e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.524798 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdb1a7-9487-40e3-ae96-a5fd41de66e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:16:55.525105 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdb1a7-9487-40e3-ae96-a5fd41de66e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:55.527108 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cdb1a7-9487-40e3-ae96-a5fd41de66e6 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:55.541103 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1404/2823] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:16:55.545505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65477d-4d7d-4986-8a3a-8b070e448bef None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:55.558462 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d65477d-4d7d-4986-8a3a-8b070e448bef None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:55.564202 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaad48-d759-4e62-b4af-74b5f3c18d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:55.564860 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaad48-d759-4e62-b4af-74b5f3c18d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.565129 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaad48-d759-4e62-b4af-74b5f3c18d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.565403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbaad48-d759-4e62-b4af-74b5f3c18d12 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.574131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbaad48-d759-4e62-b4af-74b5f3c18d12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:55.574131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbaad48-d759-4e62-b4af-74b5f3c18d12 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:55.604365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1420/2824] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:16:55 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:16:55.606572 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdb1a7-9487-40e3-ae96-a5fd41de66e6 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:55.616260 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cdb1a7-9487-40e3-ae96-a5fd41de66e6 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:55.673644 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3f2ad-034e-4fab-80c7-c26259a8ba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:55.674368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3f2ad-034e-4fab-80c7-c26259a8ba9e admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.674648 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3f2ad-034e-4fab-80c7-c26259a8ba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.674923 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3f2ad-034e-4fab-80c7-c26259a8ba9e admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.675603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3f2ad-034e-4fab-80c7-c26259a8ba9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:55.677498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3f2ad-034e-4fab-80c7-c26259a8ba9e admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:55.685140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:55.699094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1405/2825] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:16:55 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:16:55.703219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaad48-d759-4e62-b4af-74b5f3c18d12 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:55.714023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbaad48-d759-4e62-b4af-74b5f3c18d12 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:55.718773 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae763b82-7a92-43e6-8b8b-a968383f6c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:55.720052 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae763b82-7a92-43e6-8b8b-a968383f6c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.720429 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae763b82-7a92-43e6-8b8b-a968383f6c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.720782 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae763b82-7a92-43e6-8b8b-a968383f6c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.721671 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae763b82-7a92-43e6-8b8b-a968383f6c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:55.723450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae763b82-7a92-43e6-8b8b-a968383f6c0f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:55.734432 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1421/2826] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:16:55.738502 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3f2ad-034e-4fab-80c7-c26259a8ba9e admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:55.748550 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3f2ad-034e-4fab-80c7-c26259a8ba9e admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:55.753085 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a093cbed-bc8e-4e1b-8b60-b6a9cad238b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:55.753741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093cbed-bc8e-4e1b-8b60-b6a9cad238b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.754029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093cbed-bc8e-4e1b-8b60-b6a9cad238b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.754361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a093cbed-bc8e-4e1b-8b60-b6a9cad238b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.755081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093cbed-bc8e-4e1b-8b60-b6a9cad238b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:55.756925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a093cbed-bc8e-4e1b-8b60-b6a9cad238b7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:55.767467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1406/2827] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:16:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:16:55.771467 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae763b82-7a92-43e6-8b8b-a968383f6c0f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:55.781148 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae763b82-7a92-43e6-8b8b-a968383f6c0f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:55.786725 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6e5aa-a7e0-4706-bcea-16a5b42836b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:55.787586 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6e5aa-a7e0-4706-bcea-16a5b42836b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.787889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6e5aa-a7e0-4706-bcea-16a5b42836b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.788164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef6e5aa-a7e0-4706-bcea-16a5b42836b6 admin admin] PATH_INFO: `/v3/projects/60eb12fb525e4749b5c1146f75585a0c/users/3bad010bc76a435d8133811dcc1678ca/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.802541 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6e5aa-a7e0-4706-bcea-16a5b42836b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60eb12fb525e4749b5c1146f75585a0c, user_id=3bad010bc76a435d8133811dcc1678ca, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:55.805252 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef6e5aa-a7e0-4706-bcea-16a5b42836b6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:55.818259 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1422/2828] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:16:55 2022] PUT /identity/v3/projects/60eb12fb525e4749b5c1146f75585a0c/users/3bad010bc76a435d8133811dcc1678ca/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:16:55.828338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a465b-9570-47de-9378-39b92bd6f64c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.829279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a465b-9570-47de-9378-39b92bd6f64c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.829559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a465b-9570-47de-9378-39b92bd6f64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:55.855333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7b0a465b-9570-47de-9378-39b92bd6f64c None None] MFA Rules not processed for user `3bad010bc76a435d8133811dcc1678ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:16:55.856570 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7b0a465b-9570-47de-9378-39b92bd6f64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:55.920607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1407/2829] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:16:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:16:55.932561 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6e5aa-a7e0-4706-bcea-16a5b42836b6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:16:55.934305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2ef6e5aa-a7e0-4706-bcea-16a5b42836b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:16:55.973347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:16:55.991837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef6e5aa-a7e0-4706-bcea-16a5b42836b6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:16:55.995441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806df95-775c-480f-8392-7e9f4d75405a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0dc5b601421a41d1a30a10e86967a920', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:16:55.996073 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806df95-775c-480f-8392-7e9f4d75405a service glance] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:16:55.996351 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806df95-775c-480f-8392-7e9f4d75405a service glance] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:16:55.996624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806df95-775c-480f-8392-7e9f4d75405a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:16:56.005819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806df95-775c-480f-8392-7e9f4d75405a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:16:56.007225 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806df95-775c-480f-8392-7e9f4d75405a service glance] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:16:56.016952 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1423/2830] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:16:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:17:04.946969 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a465b-9570-47de-9378-39b92bd6f64c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:04.948361 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7b0a465b-9570-47de-9378-39b92bd6f64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:05.015363 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a465b-9570-47de-9378-39b92bd6f64c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:05.019912 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:05.020784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:05.021074 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:05.021378 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] PATH_INFO: `/v3/users/64150f0f08454ac3801e75bad70980ae` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:05.022604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64150f0f08454ac3801e75bad70980ae)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:05.024519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:05.074269 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64150f0f08454ac3801e75bad70980ae'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:05.075042 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64150f0f08454ac3801e75bad70980ae'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:05.075690 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64150f0f08454ac3801e75bad70980ae'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:05.088239 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64150f0f08454ac3801e75bad70980ae'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:05.095790 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64150f0f08454ac3801e75bad70980ae'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:05.097051 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:17:05.111722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1408/2831] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:17:04 2022] DELETE /identity/v3/users/64150f0f08454ac3801e75bad70980ae => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:17:05.116485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabfbd2-883b-4a88-9e6e-0d21fe3a168c None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:05.117046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabfbd2-883b-4a88-9e6e-0d21fe3a168c None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:05.117046 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aabfbd2-883b-4a88-9e6e-0d21fe3a168c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:05.134254 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-8aabfbd2-883b-4a88-9e6e-0d21fe3a168c None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:17:05.135460 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8aabfbd2-883b-4a88-9e6e-0d21fe3a168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:05.196128 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1424/2832] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:17:05.206808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:05.208286 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:05.272651 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee4e5a0-e0a5-490e-92e2-043e4c4ab493 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:05.276552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e90340-2feb-4791-b907-9a5d65416cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:05.277210 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e90340-2feb-4791-b907-9a5d65416cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:05.277496 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e90340-2feb-4791-b907-9a5d65416cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:05.277776 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e90340-2feb-4791-b907-9a5d65416cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:05.288006 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e90340-2feb-4791-b907-9a5d65416cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:05.289442 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e90340-2feb-4791-b907-9a5d65416cf8 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:05.298849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1409/2833] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:17:05.568440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabfbd2-883b-4a88-9e6e-0d21fe3a168c None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:05.569878 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-8aabfbd2-883b-4a88-9e6e-0d21fe3a168c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:05.593185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aabfbd2-883b-4a88-9e6e-0d21fe3a168c None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:05.597951 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:05.598857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:05.599135 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:05.599403 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] PATH_INFO: `/v3/projects/1cd719320fd44f72a68396014351b954` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:05.600575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cd719320fd44f72a68396014351b954)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:05.602507 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:05.650983 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cd719320fd44f72a68396014351b954', 'request_id': 'req-53500982-3ef8-4803-9ecf-1ee4b3063d18'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:05.656549 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cd719320fd44f72a68396014351b954', 'request_id': 'req-53500982-3ef8-4803-9ecf-1ee4b3063d18'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:05.664625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cd719320fd44f72a68396014351b954', 'request_id': 'req-53500982-3ef8-4803-9ecf-1ee4b3063d18'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:05.675778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] The token cache is being invalidate because project 1cd719320fd44f72a68396014351b954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:17:05.676069 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:05.676692 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-53500982-3ef8-4803-9ecf-1ee4b3063d18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:05.678260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1425/2834] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:17:05 2022] DELETE /identity/v3/projects/1cd719320fd44f72a68396014351b954 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:17:15.294695 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=31 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:17:15.672707 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:17:31.082290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e90340-2feb-4791-b907-9a5d65416cf8 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:31.083712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-20e90340-2feb-4791-b907-9a5d65416cf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:31.161476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e90340-2feb-4791-b907-9a5d65416cf8 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:31.214612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:31.215472 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:31.215754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:31.216028 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] PATH_INFO: `/v3/users/3bad010bc76a435d8133811dcc1678ca` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:31.217227 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bad010bc76a435d8133811dcc1678ca)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:31.219166 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:31.263214 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bad010bc76a435d8133811dcc1678ca'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:31.263850 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bad010bc76a435d8133811dcc1678ca'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:31.264451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bad010bc76a435d8133811dcc1678ca'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:31.276826 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bad010bc76a435d8133811dcc1678ca'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:31.283960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bad010bc76a435d8133811dcc1678ca'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:31.296138 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1410/2835] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:17:31 2022] DELETE /identity/v3/users/3bad010bc76a435d8133811dcc1678ca => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:17:31.300855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4884b78-cb01-43f2-9d67-2405a4176e83 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:31.301323 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4884b78-cb01-43f2-9d67-2405a4176e83 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:31.301422 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4884b78-cb01-43f2-9d67-2405a4176e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:31.327214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-a4884b78-cb01-43f2-9d67-2405a4176e83 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:17:31.328346 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a4884b78-cb01-43f2-9d67-2405a4176e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:31.387118 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1426/2836] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:17:31.397047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:31.398503 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:31.420404 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:17:31.493122 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41617b08-9bc7-4385-b802-70f9f9586e07 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:31.496697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638d078-76c7-4aca-9e10-1eb151a29119 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:31.497343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638d078-76c7-4aca-9e10-1eb151a29119 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:31.497624 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638d078-76c7-4aca-9e10-1eb151a29119 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:31.497896 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638d078-76c7-4aca-9e10-1eb151a29119 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:31.507569 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638d078-76c7-4aca-9e10-1eb151a29119 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:31.508986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638d078-76c7-4aca-9e10-1eb151a29119 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:31.518278 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1411/2837] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:17:31.789616 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4884b78-cb01-43f2-9d67-2405a4176e83 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:31.791188 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-a4884b78-cb01-43f2-9d67-2405a4176e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:31.803955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4884b78-cb01-43f2-9d67-2405a4176e83 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:31.808500 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:31.809453 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:31.809668 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:31.809967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] PATH_INFO: `/v3/projects/60eb12fb525e4749b5c1146f75585a0c` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:31.811318 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60eb12fb525e4749b5c1146f75585a0c)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:31.813358 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:31.860547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60eb12fb525e4749b5c1146f75585a0c', 'request_id': 'req-eb035556-239b-4089-af46-0a25e7aa38e7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:31.866944 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60eb12fb525e4749b5c1146f75585a0c', 'request_id': 'req-eb035556-239b-4089-af46-0a25e7aa38e7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:31.874675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60eb12fb525e4749b5c1146f75585a0c', 'request_id': 'req-eb035556-239b-4089-af46-0a25e7aa38e7'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:31.885739 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] The token cache is being invalidate because project 60eb12fb525e4749b5c1146f75585a0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:17:31.886047 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:31.886677 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-eb035556-239b-4089-af46-0a25e7aa38e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:17:31.888210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1427/2838] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:17:31 2022] DELETE /identity/v3/projects/60eb12fb525e4749b5c1146f75585a0c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:17:31.893035 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9db29b-d40f-4212-baa4-35dd19633cdd None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:31.893349 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9db29b-d40f-4212-baa4-35dd19633cdd None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:31.893633 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9db29b-d40f-4212-baa4-35dd19633cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:31.930855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-fb9db29b-d40f-4212-baa4-35dd19633cdd None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:17:31.932082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-fb9db29b-d40f-4212-baa4-35dd19633cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:31.990925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1412/2839] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:17:31.999749 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cddace-fc5d-4695-83b4-8c6f63fc2fe4 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:32.000039 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cddace-fc5d-4695-83b4-8c6f63fc2fe4 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:32.000321 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cddace-fc5d-4695-83b4-8c6f63fc2fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:32.018444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-07cddace-fc5d-4695-83b4-8c6f63fc2fe4 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:17:32.019643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-07cddace-fc5d-4695-83b4-8c6f63fc2fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:32.027448 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1428/2840] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:17:32.031429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9db29b-d40f-4212-baa4-35dd19633cdd None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:32.041376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9db29b-d40f-4212-baa4-35dd19633cdd None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:32.045895 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555b91f-3521-4e2a-b61d-586f6eceb019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:32.046582 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555b91f-3521-4e2a-b61d-586f6eceb019 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:32.046867 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555b91f-3521-4e2a-b61d-586f6eceb019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:32.047143 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6555b91f-3521-4e2a-b61d-586f6eceb019 admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:32.047866 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555b91f-3521-4e2a-b61d-586f6eceb019 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:17:32.048173 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555b91f-3521-4e2a-b61d-586f6eceb019 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:32.050454 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6555b91f-3521-4e2a-b61d-586f6eceb019 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:32.061707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1413/2841] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:17:32.065799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cddace-fc5d-4695-83b4-8c6f63fc2fe4 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:32.074760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cddace-fc5d-4695-83b4-8c6f63fc2fe4 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:32.079354 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8a92e-dc7a-417e-b0de-2a51db63d90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:32.079996 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8a92e-dc7a-417e-b0de-2a51db63d90b admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:32.080275 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8a92e-dc7a-417e-b0de-2a51db63d90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:32.080546 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b8a92e-dc7a-417e-b0de-2a51db63d90b admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:32.081249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8a92e-dc7a-417e-b0de-2a51db63d90b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:32.083185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b8a92e-dc7a-417e-b0de-2a51db63d90b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:32.107826 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1429/2842] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:17:32 2022] POST /identity/v3/projects => generated 371 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:17:32.112534 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555b91f-3521-4e2a-b61d-586f6eceb019 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:32.122407 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6555b91f-3521-4e2a-b61d-586f6eceb019 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:32.172603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:17:32.178761 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e8c86-3bd4-4245-a85b-ebcbefa83dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:32.179431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e8c86-3bd4-4245-a85b-ebcbefa83dfc admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:32.179720 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e8c86-3bd4-4245-a85b-ebcbefa83dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:32.180001 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8e8c86-3bd4-4245-a85b-ebcbefa83dfc admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:32.180697 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e8c86-3bd4-4245-a85b-ebcbefa83dfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:32.182643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8e8c86-3bd4-4245-a85b-ebcbefa83dfc admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:32.201672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1414/2843] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:17:32 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:17:32.206329 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8a92e-dc7a-417e-b0de-2a51db63d90b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:32.216320 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b8a92e-dc7a-417e-b0de-2a51db63d90b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:32.220858 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad3f56-0c4f-4cce-aa2e-60dc3b7d8062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:32.222192 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad3f56-0c4f-4cce-aa2e-60dc3b7d8062 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:32.222485 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad3f56-0c4f-4cce-aa2e-60dc3b7d8062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:32.222763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad3f56-0c4f-4cce-aa2e-60dc3b7d8062 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:32.223548 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad3f56-0c4f-4cce-aa2e-60dc3b7d8062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:32.225628 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad3f56-0c4f-4cce-aa2e-60dc3b7d8062 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:32.238412 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1430/2844] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:17:32.242493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e8c86-3bd4-4245-a85b-ebcbefa83dfc admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:32.252560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8e8c86-3bd4-4245-a85b-ebcbefa83dfc admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:32.257477 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f444190-8e56-42ee-ae33-52ddacbd1081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:32.258192 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f444190-8e56-42ee-ae33-52ddacbd1081 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:32.258479 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f444190-8e56-42ee-ae33-52ddacbd1081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:32.258760 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f444190-8e56-42ee-ae33-52ddacbd1081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:32.259485 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f444190-8e56-42ee-ae33-52ddacbd1081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:32.261064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f444190-8e56-42ee-ae33-52ddacbd1081 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:32.271549 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1415/2845] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:17:32.275440 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad3f56-0c4f-4cce-aa2e-60dc3b7d8062 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:32.285786 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad3f56-0c4f-4cce-aa2e-60dc3b7d8062 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:32.290344 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45bcf9-74cd-4ba2-8522-206fb9d68899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:32.291207 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45bcf9-74cd-4ba2-8522-206fb9d68899 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:32.291486 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45bcf9-74cd-4ba2-8522-206fb9d68899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:32.291756 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb45bcf9-74cd-4ba2-8522-206fb9d68899 admin admin] PATH_INFO: `/v3/projects/80a33a31ba6046e78352f5ba9e88b9fe/users/faa6dfc6e5c14046b499354e0f07a404/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:32.305685 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45bcf9-74cd-4ba2-8522-206fb9d68899 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a33a31ba6046e78352f5ba9e88b9fe, user_id=faa6dfc6e5c14046b499354e0f07a404, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:32.308325 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb45bcf9-74cd-4ba2-8522-206fb9d68899 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:32.324889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1431/2846] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:17:32 2022] PUT /identity/v3/projects/80a33a31ba6046e78352f5ba9e88b9fe/users/faa6dfc6e5c14046b499354e0f07a404/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:17:32.326676 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91da675-9109-4dbd-8fed-632ecb517e4b None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:32.326973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91da675-9109-4dbd-8fed-632ecb517e4b None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:32.327253 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91da675-9109-4dbd-8fed-632ecb517e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:32.344844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-b91da675-9109-4dbd-8fed-632ecb517e4b None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:17:32.346034 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-b91da675-9109-4dbd-8fed-632ecb517e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:32.402745 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1416/2847] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:17:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:17:32.412625 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45bcf9-74cd-4ba2-8522-206fb9d68899 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:32.414306 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-eb45bcf9-74cd-4ba2-8522-206fb9d68899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:32.428571 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:17:32.472551 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb45bcf9-74cd-4ba2-8522-206fb9d68899 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:32.476153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060ff18-1195-40e6-b30a-7e162682a778 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:32.476784 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060ff18-1195-40e6-b30a-7e162682a778 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:32.477064 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060ff18-1195-40e6-b30a-7e162682a778 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:32.477337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1060ff18-1195-40e6-b30a-7e162682a778 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:32.486110 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060ff18-1195-40e6-b30a-7e162682a778 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:32.487544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1060ff18-1195-40e6-b30a-7e162682a778 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:32.496643 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1432/2848] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:17:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:17:36.131750 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e422dd-0adc-4d28-8607-2d0a2088d30c None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:36.132265 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e422dd-0adc-4d28-8607-2d0a2088d30c None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:36.132407 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e422dd-0adc-4d28-8607-2d0a2088d30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:36.159994 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-79e422dd-0adc-4d28-8607-2d0a2088d30c None None] MFA Rules not processed for user `faa6dfc6e5c14046b499354e0f07a404`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:17:36.161247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-79e422dd-0adc-4d28-8607-2d0a2088d30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:36.230962 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1417/2849] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:17:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:17:36.235533 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060ff18-1195-40e6-b30a-7e162682a778 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:36.246086 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1060ff18-1195-40e6-b30a-7e162682a778 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:36.250800 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735817c-5276-4540-a8e7-26d7654f40b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:36.251604 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735817c-5276-4540-a8e7-26d7654f40b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:36.251894 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735817c-5276-4540-a8e7-26d7654f40b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:36.252177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735817c-5276-4540-a8e7-26d7654f40b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:36.253568 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735817c-5276-4540-a8e7-26d7654f40b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:36.255543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735817c-5276-4540-a8e7-26d7654f40b0 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:36.286236 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1433/2850] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:17:36 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:17:36.287515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e422dd-0adc-4d28-8607-2d0a2088d30c None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:36.302247 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e422dd-0adc-4d28-8607-2d0a2088d30c None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:36.360892 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1871f-5ab9-4b78-bd9e-b560fa5ef77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:36.361573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1871f-5ab9-4b78-bd9e-b560fa5ef77d admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:36.361913 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1871f-5ab9-4b78-bd9e-b560fa5ef77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:36.362723 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1871f-5ab9-4b78-bd9e-b560fa5ef77d admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:36.363409 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1871f-5ab9-4b78-bd9e-b560fa5ef77d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:36.365321 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1871f-5ab9-4b78-bd9e-b560fa5ef77d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:36.375664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:17:36.387808 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1418/2851] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:17:36 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:17:36.391871 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735817c-5276-4540-a8e7-26d7654f40b0 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:36.402547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735817c-5276-4540-a8e7-26d7654f40b0 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:36.407476 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee0da0-6fc3-48b9-b71e-6da0a0967b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:36.408196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee0da0-6fc3-48b9-b71e-6da0a0967b67 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:36.408479 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee0da0-6fc3-48b9-b71e-6da0a0967b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:36.409280 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ee0da0-6fc3-48b9-b71e-6da0a0967b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:36.410007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee0da0-6fc3-48b9-b71e-6da0a0967b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:36.411659 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ee0da0-6fc3-48b9-b71e-6da0a0967b67 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:36.423636 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1434/2852] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:17:36.428555 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1871f-5ab9-4b78-bd9e-b560fa5ef77d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:36.439215 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1871f-5ab9-4b78-bd9e-b560fa5ef77d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:36.444064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070adad-6141-4ead-8eaf-6a9526f2b680 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:36.444969 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070adad-6141-4ead-8eaf-6a9526f2b680 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:36.445254 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070adad-6141-4ead-8eaf-6a9526f2b680 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:36.445539 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9070adad-6141-4ead-8eaf-6a9526f2b680 admin admin] PATH_INFO: `/v3/projects/2fb02fd66e6c4d51be2e1b77085cf32d/users/02c3f36cb4c142a494041163cc76ae79/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:36.461152 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070adad-6141-4ead-8eaf-6a9526f2b680 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb02fd66e6c4d51be2e1b77085cf32d, user_id=02c3f36cb4c142a494041163cc76ae79, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:36.463869 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9070adad-6141-4ead-8eaf-6a9526f2b680 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:36.479498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1419/2853] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:17:36 2022] PUT /identity/v3/projects/2fb02fd66e6c4d51be2e1b77085cf32d/users/02c3f36cb4c142a494041163cc76ae79/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:17:36.483337 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee0da0-6fc3-48b9-b71e-6da0a0967b67 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:36.494157 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ee0da0-6fc3-48b9-b71e-6da0a0967b67 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:36.552131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc2db06-88b4-4dc2-9958-e2a57feeeadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:36.552808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2db06-88b4-4dc2-9958-e2a57feeeadd admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:36.553098 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2db06-88b4-4dc2-9958-e2a57feeeadd admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:36.553378 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc2db06-88b4-4dc2-9958-e2a57feeeadd admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:36.554136 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2db06-88b4-4dc2-9958-e2a57feeeadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:36.555727 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc2db06-88b4-4dc2-9958-e2a57feeeadd admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:36.567519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1435/2854] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:17:36.570487 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070adad-6141-4ead-8eaf-6a9526f2b680 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:36.581042 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9070adad-6141-4ead-8eaf-6a9526f2b680 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:36.585674 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447352c-dad9-42d3-bffd-a960b82e3f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:36.586613 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447352c-dad9-42d3-bffd-a960b82e3f57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:36.586908 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447352c-dad9-42d3-bffd-a960b82e3f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:36.587171 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447352c-dad9-42d3-bffd-a960b82e3f57 admin admin] PATH_INFO: `/v3/projects/2fb02fd66e6c4d51be2e1b77085cf32d/users/02c3f36cb4c142a494041163cc76ae79/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:36.589165 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447352c-dad9-42d3-bffd-a960b82e3f57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb02fd66e6c4d51be2e1b77085cf32d, user_id=02c3f36cb4c142a494041163cc76ae79, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:36.591858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447352c-dad9-42d3-bffd-a960b82e3f57 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:36.603134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1420/2855] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:17:36 2022] PUT /identity/v3/projects/2fb02fd66e6c4d51be2e1b77085cf32d/users/02c3f36cb4c142a494041163cc76ae79/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:17:41.263139 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c51ef-4802-4dd6-b86d-8d1ecb3e16f3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:41.265094 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c51ef-4802-4dd6-b86d-8d1ecb3e16f3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:41.265550 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1c51ef-4802-4dd6-b86d-8d1ecb3e16f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:41.294763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-bd1c51ef-4802-4dd6-b86d-8d1ecb3e16f3 None None] MFA Rules not processed for user `02c3f36cb4c142a494041163cc76ae79`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:17:41.296000 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-bd1c51ef-4802-4dd6-b86d-8d1ecb3e16f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:41.367386 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1436/2856] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:17:41 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:17:41.384208 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447352c-dad9-42d3-bffd-a960b82e3f57 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:17:41.386875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7447352c-dad9-42d3-bffd-a960b82e3f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:41.472136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447352c-dad9-42d3-bffd-a960b82e3f57 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:17:41.477616 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc675b7-9758-4c18-a6a4-f43975cf2c1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:17:41.478300 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc675b7-9758-4c18-a6a4-f43975cf2c1b service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:41.478592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc675b7-9758-4c18-a6a4-f43975cf2c1b service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:41.478873 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc675b7-9758-4c18-a6a4-f43975cf2c1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:41.540433 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc675b7-9758-4c18-a6a4-f43975cf2c1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:17:41.542163 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc675b7-9758-4c18-a6a4-f43975cf2c1b service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:17:41.553098 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1421/2857] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:17:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:17:43.627314 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ec518-ba74-405f-bc70-37e8c35c5b02 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:17:43.627954 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ec518-ba74-405f-bc70-37e8c35c5b02 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:17:43.628229 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ec518-ba74-405f-bc70-37e8c35c5b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:17:43.635906 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:17:43.648622 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-084ec518-ba74-405f-bc70-37e8c35c5b02 None None] MFA Rules not processed for user `faa6dfc6e5c14046b499354e0f07a404`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:17:43.649831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-084ec518-ba74-405f-bc70-37e8c35c5b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:17:43.657543 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1437/2858] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:17:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:17:51.547857 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=73,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:17:53.645891 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:18:05.397313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc675b7-9758-4c18-a6a4-f43975cf2c1b service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:05.449872 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc675b7-9758-4c18-a6a4-f43975cf2c1b service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:05.453634 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf60f7c-bc8c-4eed-aecb-4cb8d73c9b6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:05.454334 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf60f7c-bc8c-4eed-aecb-4cb8d73c9b6f service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:05.454626 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf60f7c-bc8c-4eed-aecb-4cb8d73c9b6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:05.454907 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf60f7c-bc8c-4eed-aecb-4cb8d73c9b6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:05.456664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ecf60f7c-bc8c-4eed-aecb-4cb8d73c9b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:05.466140 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf60f7c-bc8c-4eed-aecb-4cb8d73c9b6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:05.467557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf60f7c-bc8c-4eed-aecb-4cb8d73c9b6f service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:05.475960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1422/2859] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:18:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:18:05.930339 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ec518-ba74-405f-bc70-37e8c35c5b02 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:05.931803 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-084ec518-ba74-405f-bc70-37e8c35c5b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:06.023522 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ec518-ba74-405f-bc70-37e8c35c5b02 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:06.028364 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7d3f3-5dce-4e63-8137-282ec2ad76a8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:06.029002 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7d3f3-5dce-4e63-8137-282ec2ad76a8 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:06.029283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7d3f3-5dce-4e63-8137-282ec2ad76a8 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:06.029555 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c7d3f3-5dce-4e63-8137-282ec2ad76a8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:06.031317 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-70c7d3f3-5dce-4e63-8137-282ec2ad76a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:06.043353 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7d3f3-5dce-4e63-8137-282ec2ad76a8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:06.044929 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c7d3f3-5dce-4e63-8137-282ec2ad76a8 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:06.054536 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1438/2860] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:18:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:18:06.496313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf60f7c-bc8c-4eed-aecb-4cb8d73c9b6f service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:06.497711 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ecf60f7c-bc8c-4eed-aecb-4cb8d73c9b6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:06.512449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf60f7c-bc8c-4eed-aecb-4cb8d73c9b6f service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:06.517408 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e2a4c-6fce-4a47-ae34-8d633dbc1450 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:06.518075 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2a4c-6fce-4a47-ae34-8d633dbc1450 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:06.518360 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2a4c-6fce-4a47-ae34-8d633dbc1450 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:06.518640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970e2a4c-6fce-4a47-ae34-8d633dbc1450 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:06.520382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-970e2a4c-6fce-4a47-ae34-8d633dbc1450 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:06.531592 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e2a4c-6fce-4a47-ae34-8d633dbc1450 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:06.533144 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970e2a4c-6fce-4a47-ae34-8d633dbc1450 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:06.543515 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1423/2861] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:18:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:18:08.639221 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7d3f3-5dce-4e63-8137-282ec2ad76a8 service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:08.641672 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-70c7d3f3-5dce-4e63-8137-282ec2ad76a8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:08.671144 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c7d3f3-5dce-4e63-8137-282ec2ad76a8 service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:08.677680 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf87e21-6256-4711-9b30-e81b71a5dd82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:08.679289 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf87e21-6256-4711-9b30-e81b71a5dd82 service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:08.679566 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf87e21-6256-4711-9b30-e81b71a5dd82 service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:08.679851 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf87e21-6256-4711-9b30-e81b71a5dd82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:08.682130 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3bf87e21-6256-4711-9b30-e81b71a5dd82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:08.696307 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf87e21-6256-4711-9b30-e81b71a5dd82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:08.698059 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf87e21-6256-4711-9b30-e81b71a5dd82 service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:08.718819 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1439/2862] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:18:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:18:15.688546 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e2a4c-6fce-4a47-ae34-8d633dbc1450 service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:15.702261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970e2a4c-6fce-4a47-ae34-8d633dbc1450 service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:15.707718 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca771c1a-6806-4120-b5c6-5702947a71ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:15.708387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca771c1a-6806-4120-b5c6-5702947a71ea service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:15.708707 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca771c1a-6806-4120-b5c6-5702947a71ea service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:15.708982 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca771c1a-6806-4120-b5c6-5702947a71ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:15.710755 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ca771c1a-6806-4120-b5c6-5702947a71ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:15.723440 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca771c1a-6806-4120-b5c6-5702947a71ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:15.725000 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca771c1a-6806-4120-b5c6-5702947a71ea service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:15.736919 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1424/2863] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:18:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:18:18.705683 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=47 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:18:20.740095 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf87e21-6256-4711-9b30-e81b71a5dd82 service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:20.752179 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf87e21-6256-4711-9b30-e81b71a5dd82 service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:20.758196 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f132f0c-9149-4e16-aa6b-b5bd6cad9a4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:20.758843 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f132f0c-9149-4e16-aa6b-b5bd6cad9a4b service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:20.759131 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f132f0c-9149-4e16-aa6b-b5bd6cad9a4b service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:20.759416 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f132f0c-9149-4e16-aa6b-b5bd6cad9a4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:20.761214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9f132f0c-9149-4e16-aa6b-b5bd6cad9a4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:20.775562 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f132f0c-9149-4e16-aa6b-b5bd6cad9a4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:20.777146 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f132f0c-9149-4e16-aa6b-b5bd6cad9a4b service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:20.788347 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1440/2864] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:18:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:18:23.887047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca771c1a-6806-4120-b5c6-5702947a71ea service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:23.888446 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ca771c1a-6806-4120-b5c6-5702947a71ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:23.899864 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca771c1a-6806-4120-b5c6-5702947a71ea service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:23.903326 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ff3e9-5b42-4b31-81cb-991855a43e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:23.903955 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ff3e9-5b42-4b31-81cb-991855a43e72 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:23.904240 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ff3e9-5b42-4b31-81cb-991855a43e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:23.904519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ff3e9-5b42-4b31-81cb-991855a43e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:23.913250 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ff3e9-5b42-4b31-81cb-991855a43e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:23.914724 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ff3e9-5b42-4b31-81cb-991855a43e72 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:23.923938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1425/2865] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:18:23 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:18:24.054012 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e1cae-03c5-41d6-ad63-ca688c7373d3 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:24.054491 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e1cae-03c5-41d6-ad63-ca688c7373d3 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:24.054612 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e1cae-03c5-41d6-ad63-ca688c7373d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:24.073575 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-f84e1cae-03c5-41d6-ad63-ca688c7373d3 None None] MFA Rules not processed for user `828fd11b79d045dc85e0973cd255bcf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:18:24.074857 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f84e1cae-03c5-41d6-ad63-ca688c7373d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:24.134927 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1441/2866] 10.222.0.13 () {58 vars in 967 bytes} [Sun Jun 26 03:18:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:18:24.175045 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ff3e9-5b42-4b31-81cb-991855a43e72 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:24.176987 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-407ff3e9-5b42-4b31-81cb-991855a43e72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:24.247153 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ff3e9-5b42-4b31-81cb-991855a43e72 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:24.252325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befde777-7005-455c-89d2-986166c4b472 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '60f006c99809427496b026c17a663995', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:24.253344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befde777-7005-455c-89d2-986166c4b472 service swift] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:24.253344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befde777-7005-455c-89d2-986166c4b472 service swift] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:24.253552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befde777-7005-455c-89d2-986166c4b472 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:24.262198 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befde777-7005-455c-89d2-986166c4b472 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:24.263844 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befde777-7005-455c-89d2-986166c4b472 service swift] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:24.272113 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1426/2867] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Jun 26 03:18:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 26 03:18:27.719376 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e1cae-03c5-41d6-ad63-ca688c7373d3 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:27.737248 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e1cae-03c5-41d6-ad63-ca688c7373d3 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:27.750172 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb3099-a3fb-4f03-8a4a-4b967dc5f6ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:27.750919 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb3099-a3fb-4f03-8a4a-4b967dc5f6ab service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:27.751270 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb3099-a3fb-4f03-8a4a-4b967dc5f6ab service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:27.751637 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb3099-a3fb-4f03-8a4a-4b967dc5f6ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:27.754126 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-2afb3099-a3fb-4f03-8a4a-4b967dc5f6ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:27.772404 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb3099-a3fb-4f03-8a4a-4b967dc5f6ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:27.774396 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb3099-a3fb-4f03-8a4a-4b967dc5f6ab service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:27.787501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1442/2868] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:18:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:18:30.849661 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befde777-7005-455c-89d2-986166c4b472 service swift] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:30.852344 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-befde777-7005-455c-89d2-986166c4b472 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:30.913420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=100 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:18:30.928236 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befde777-7005-455c-89d2-986166c4b472 service swift] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:30.933136 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:30.933731 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:30.934154 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:30.934463 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] PATH_INFO: `/v3/users/1042fb5384e14cd2bc7090f09db977c2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:30.935664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1042fb5384e14cd2bc7090f09db977c2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:30.937587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:30.990803 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1042fb5384e14cd2bc7090f09db977c2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:30.991476 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1042fb5384e14cd2bc7090f09db977c2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:30.992090 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1042fb5384e14cd2bc7090f09db977c2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.005598 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1042fb5384e14cd2bc7090f09db977c2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.014523 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1042fb5384e14cd2bc7090f09db977c2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.028290 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1427/2869] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:18:30 2022] DELETE /identity/v3/users/1042fb5384e14cd2bc7090f09db977c2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:18:31.201544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb3099-a3fb-4f03-8a4a-4b967dc5f6ab service placement] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:31.213040 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb3099-a3fb-4f03-8a4a-4b967dc5f6ab service placement] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:31.274828 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:31.275689 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:31.275972 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:31.276250 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] PATH_INFO: `/v3/projects/1fe0cd601688420d88ea051894cfdf2a` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:31.277423 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe0cd601688420d88ea051894cfdf2a)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:31.279349 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:31.323090 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe0cd601688420d88ea051894cfdf2a', 'request_id': 'req-789a0561-95af-4f37-a626-88914c75f92b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.328641 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe0cd601688420d88ea051894cfdf2a', 'request_id': 'req-789a0561-95af-4f37-a626-88914c75f92b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.337023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe0cd601688420d88ea051894cfdf2a', 'request_id': 'req-789a0561-95af-4f37-a626-88914c75f92b'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.347704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] The token cache is being invalidate because project 1fe0cd601688420d88ea051894cfdf2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:18:31.347990 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.348620 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.350152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1443/2870] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:18:31 2022] DELETE /identity/v3/projects/1fe0cd601688420d88ea051894cfdf2a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:18:31.354226 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:31.355625 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:31.369206 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd65ac7e-132f-4571-8329-510f30b7fe00 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:31.423821 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:31.424747 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:31.424973 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:31.425849 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] PATH_INFO: `/v3/users/f907c9b48f8141788f3fad618dc545ab` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:31.426963 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f907c9b48f8141788f3fad618dc545ab)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:31.428866 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:31.472448 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f907c9b48f8141788f3fad618dc545ab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.473088 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f907c9b48f8141788f3fad618dc545ab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.473713 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f907c9b48f8141788f3fad618dc545ab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.485323 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f907c9b48f8141788f3fad618dc545ab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.493058 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-95106551-3eb1-4975-9adf-5754fb360433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f907c9b48f8141788f3fad618dc545ab'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.495029 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:18:31.511597 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1428/2871] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:18:31 2022] DELETE /identity/v3/users/f907c9b48f8141788f3fad618dc545ab => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:18:31.679943 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:31.681389 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:31.698106 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a0561-95af-4f37-a626-88914c75f92b admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:31.703099 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:18:31.759297 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:31.760151 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:31.760435 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:31.760714 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] PATH_INFO: `/v3/projects/7068039fead9452e89458bf2ea06dfeb` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:31.761885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7068039fead9452e89458bf2ea06dfeb)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:31.763835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:31.809164 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7068039fead9452e89458bf2ea06dfeb', 'request_id': 'req-d6579e12-e1cf-4cc3-9cce-e02055373a8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.814646 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7068039fead9452e89458bf2ea06dfeb', 'request_id': 'req-d6579e12-e1cf-4cc3-9cce-e02055373a8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.821517 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7068039fead9452e89458bf2ea06dfeb', 'request_id': 'req-d6579e12-e1cf-4cc3-9cce-e02055373a8c'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.832232 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] The token cache is being invalidate because project 7068039fead9452e89458bf2ea06dfeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:18:31.832530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.833163 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d6579e12-e1cf-4cc3-9cce-e02055373a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:18:31.834743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1444/2872] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:18:31 2022] DELETE /identity/v3/projects/7068039fead9452e89458bf2ea06dfeb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:18:31.839817 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef16ec-34f1-42e1-ab78-c693f040a4b2 None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:31.840111 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef16ec-34f1-42e1-ab78-c693f040a4b2 None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:31.840392 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ef16ec-34f1-42e1-ab78-c693f040a4b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:31.858929 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-f5ef16ec-34f1-42e1-ab78-c693f040a4b2 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:18:31.860125 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-f5ef16ec-34f1-42e1-ab78-c693f040a4b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:31.928335 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1429/2873] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:18:31.935700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef729627-e4b8-4f7f-adc0-a9df08506c39 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:31.936007 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef729627-e4b8-4f7f-adc0-a9df08506c39 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:31.936283 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef729627-e4b8-4f7f-adc0-a9df08506c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:31.954365 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-ef729627-e4b8-4f7f-adc0-a9df08506c39 None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:18:31.955583 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-ef729627-e4b8-4f7f-adc0-a9df08506c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:31.963565 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1445/2874] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:18:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:18:31.967612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef16ec-34f1-42e1-ab78-c693f040a4b2 None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:31.977612 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ef16ec-34f1-42e1-ab78-c693f040a4b2 None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:31.982187 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db598a-06c2-4828-ae0d-ca26bfc5201f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:31.982946 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db598a-06c2-4828-ae0d-ca26bfc5201f admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:31.983124 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db598a-06c2-4828-ae0d-ca26bfc5201f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:31.983400 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4db598a-06c2-4828-ae0d-ca26bfc5201f admin admin] PATH_INFO: `/v3/domains` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:31.984121 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db598a-06c2-4828-ae0d-ca26bfc5201f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80103) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 26 03:18:31.984429 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db598a-06c2-4828-ae0d-ca26bfc5201f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:31.986133 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4db598a-06c2-4828-ae0d-ca26bfc5201f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:31.998420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1430/2875] 10.222.0.13 () {58 vars in 1152 bytes} [Sun Jun 26 03:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 26 03:18:32.002655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef729627-e4b8-4f7f-adc0-a9df08506c39 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:32.012152 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef729627-e4b8-4f7f-adc0-a9df08506c39 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:32.016980 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865da88-5171-49b5-bf07-6a15be6cdf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:32.017732 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865da88-5171-49b5-bf07-6a15be6cdf9a admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:32.018021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865da88-5171-49b5-bf07-6a15be6cdf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:32.018331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865da88-5171-49b5-bf07-6a15be6cdf9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:32.019033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865da88-5171-49b5-bf07-6a15be6cdf9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:32.021004 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865da88-5171-49b5-bf07-6a15be6cdf9a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:32.046693 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1446/2876] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:18:32 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 26 03:18:32.051308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db598a-06c2-4828-ae0d-ca26bfc5201f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:32.061099 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4db598a-06c2-4828-ae0d-ca26bfc5201f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:32.116376 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f3d83-3b93-47c8-9666-67c238cee2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:32.117038 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f3d83-3b93-47c8-9666-67c238cee2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:32.117325 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f3d83-3b93-47c8-9666-67c238cee2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:32.117603 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45f3d83-3b93-47c8-9666-67c238cee2e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:32.118318 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f3d83-3b93-47c8-9666-67c238cee2e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:32.120218 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45f3d83-3b93-47c8-9666-67c238cee2e5 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:32.139589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1431/2877] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:18:32 2022] POST /identity/v3/users => generated 290 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:18:32.144518 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865da88-5171-49b5-bf07-6a15be6cdf9a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:32.155281 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865da88-5171-49b5-bf07-6a15be6cdf9a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:32.160446 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19715161-f5e8-4dfc-93a8-5266b3c56ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:32.161092 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19715161-f5e8-4dfc-93a8-5266b3c56ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:32.161374 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19715161-f5e8-4dfc-93a8-5266b3c56ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:32.161648 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19715161-f5e8-4dfc-93a8-5266b3c56ee6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:32.162520 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19715161-f5e8-4dfc-93a8-5266b3c56ee6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:32.164102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19715161-f5e8-4dfc-93a8-5266b3c56ee6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:32.174500 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1447/2878] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:18:32.178156 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f3d83-3b93-47c8-9666-67c238cee2e5 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:32.188308 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45f3d83-3b93-47c8-9666-67c238cee2e5 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:32.193410 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5ee610-752b-48e0-bdf8-a160c109f883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:32.194331 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ee610-752b-48e0-bdf8-a160c109f883 admin admin] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:32.194615 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ee610-752b-48e0-bdf8-a160c109f883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:32.194888 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5ee610-752b-48e0-bdf8-a160c109f883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:32.195607 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ee610-752b-48e0-bdf8-a160c109f883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:32.197165 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5ee610-752b-48e0-bdf8-a160c109f883 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:32.208120 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1432/2879] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:18:32.212003 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19715161-f5e8-4dfc-93a8-5266b3c56ee6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:32.221728 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19715161-f5e8-4dfc-93a8-5266b3c56ee6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:32.226823 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df52c36-14c7-4b32-9f6c-36ac34189c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:32.227682 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df52c36-14c7-4b32-9f6c-36ac34189c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:32.227958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df52c36-14c7-4b32-9f6c-36ac34189c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:32.228227 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df52c36-14c7-4b32-9f6c-36ac34189c54 admin admin] PATH_INFO: `/v3/projects/4a6b41994f444cafb21d3d6e395b6893/users/6d52b7f924f24f518ad0c9dfb6008260/roles/e1ed407e2d924fac8012250bb2ecf94f` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:32.242218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df52c36-14c7-4b32-9f6c-36ac34189c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6b41994f444cafb21d3d6e395b6893, user_id=6d52b7f924f24f518ad0c9dfb6008260, role_id=e1ed407e2d924fac8012250bb2ecf94f)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:32.244805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df52c36-14c7-4b32-9f6c-36ac34189c54 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:32.257406 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1448/2880] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:18:32 2022] PUT /identity/v3/projects/4a6b41994f444cafb21d3d6e395b6893/users/6d52b7f924f24f518ad0c9dfb6008260/roles/e1ed407e2d924fac8012250bb2ecf94f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 26 03:18:32.262345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0dc4b-bbe3-4865-ab57-6078b11834fd None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:32.262670 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0dc4b-bbe3-4865-ab57-6078b11834fd None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:32.262953 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0dc4b-bbe3-4865-ab57-6078b11834fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:32.281047 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-20d0dc4b-bbe3-4865-ab57-6078b11834fd None None] MFA Rules not processed for user `a05c37bdf694480a8a14de0b18b2da4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:18:32.282322 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-20d0dc4b-bbe3-4865-ab57-6078b11834fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:32.329696 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:18:32.340440 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1433/2881] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:18:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:18:32.350371 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df52c36-14c7-4b32-9f6c-36ac34189c54 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:32.352050 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-9df52c36-14c7-4b32-9f6c-36ac34189c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:32.390437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:18:32.410243 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df52c36-14c7-4b32-9f6c-36ac34189c54 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:32.413835 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7944c-bd9e-41e7-83f6-38241af97044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:32.414513 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7944c-bd9e-41e7-83f6-38241af97044 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:32.414798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7944c-bd9e-41e7-83f6-38241af97044 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:32.415070 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c7944c-bd9e-41e7-83f6-38241af97044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:32.423876 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7944c-bd9e-41e7-83f6-38241af97044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:32.425295 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c7944c-bd9e-41e7-83f6-38241af97044 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:32.434532 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1449/2882] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:18:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:18:36.564584 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287a72-bf35-48a4-b252-592529601a1d None None] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:36.565162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287a72-bf35-48a4-b252-592529601a1d None None] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:36.565162 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a287a72-bf35-48a4-b252-592529601a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:36.595298 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.auth.core [None req-7a287a72-bf35-48a4-b252-592529601a1d None None] MFA Rules not processed for user `6d52b7f924f24f518ad0c9dfb6008260`. Rule list: `[]` (Enabled: `True`). {{(pid=80103) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:18:36.596589 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7a287a72-bf35-48a4-b252-592529601a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:36.674175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1434/2883] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:18:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 26 03:18:36.679195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7944c-bd9e-41e7-83f6-38241af97044 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:36.689505 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c7944c-bd9e-41e7-83f6-38241af97044 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:36.694789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bd56f-2512-4551-812a-3adf622336a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:36.695475 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bd56f-2512-4551-812a-3adf622336a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:36.695763 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bd56f-2512-4551-812a-3adf622336a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:36.696041 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26bd56f-2512-4551-812a-3adf622336a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:36.696740 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bd56f-2512-4551-812a-3adf622336a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:36.698675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26bd56f-2512-4551-812a-3adf622336a6 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:36.723981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1450/2884] 10.222.0.13 () {60 vars in 1152 bytes} [Sun Jun 26 03:18:36 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:18:36.728332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a287a72-bf35-48a4-b252-592529601a1d None None] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:36.740875 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a287a72-bf35-48a4-b252-592529601a1d None None] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:36.808064 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a74ae-6b1d-4987-9843-1fc1cfbf4956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:36.808744 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a74ae-6b1d-4987-9843-1fc1cfbf4956 admin admin] REQUEST_METHOD: `POST` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:36.809078 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a74ae-6b1d-4987-9843-1fc1cfbf4956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:36.809330 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a74ae-6b1d-4987-9843-1fc1cfbf4956 admin admin] PATH_INFO: `/v3/users` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:36.810027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a74ae-6b1d-4987-9843-1fc1cfbf4956 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:36.812095 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a74ae-6b1d-4987-9843-1fc1cfbf4956 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:36.835018 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1435/2885] 10.222.0.13 () {60 vars in 1143 bytes} [Sun Jun 26 03:18:36 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 26 03:18:36.839141 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bd56f-2512-4551-812a-3adf622336a6 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:36.849674 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26bd56f-2512-4551-812a-3adf622336a6 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:36.854902 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe86d2-ca7c-4e49-ba34-dd646bfa1660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:36.855731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe86d2-ca7c-4e49-ba34-dd646bfa1660 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:36.856027 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe86d2-ca7c-4e49-ba34-dd646bfa1660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:36.856244 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe86d2-ca7c-4e49-ba34-dd646bfa1660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:36.856982 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe86d2-ca7c-4e49-ba34-dd646bfa1660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:36.858624 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe86d2-ca7c-4e49-ba34-dd646bfa1660 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:36.872808 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1451/2886] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 26 03:18:36.876604 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a74ae-6b1d-4987-9843-1fc1cfbf4956 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:36.888387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a74ae-6b1d-4987-9843-1fc1cfbf4956 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:36.893011 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05f3ad-33ce-4fa4-8d1c-6d1fbee2a1c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:36.893921 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05f3ad-33ce-4fa4-8d1c-6d1fbee2a1c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:36.894241 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05f3ad-33ce-4fa4-8d1c-6d1fbee2a1c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:36.894518 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c05f3ad-33ce-4fa4-8d1c-6d1fbee2a1c7 admin admin] PATH_INFO: `/v3/projects/ab9dc6af1a444a42ab539b5be5fc1922/users/fd58e94ef99d4ce7ba4b860b082bbfe2/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:36.910382 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05f3ad-33ce-4fa4-8d1c-6d1fbee2a1c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9dc6af1a444a42ab539b5be5fc1922, user_id=fd58e94ef99d4ce7ba4b860b082bbfe2, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:36.913079 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c05f3ad-33ce-4fa4-8d1c-6d1fbee2a1c7 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:36.927986 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1436/2887] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:18:36 2022] PUT /identity/v3/projects/ab9dc6af1a444a42ab539b5be5fc1922/users/fd58e94ef99d4ce7ba4b860b082bbfe2/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:18:36.932045 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe86d2-ca7c-4e49-ba34-dd646bfa1660 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:36.943955 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe86d2-ca7c-4e49-ba34-dd646bfa1660 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:37.006010 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32078e98-66db-40b2-a466-4fbfe7268911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:37.006704 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32078e98-66db-40b2-a466-4fbfe7268911 admin admin] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:37.007240 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32078e98-66db-40b2-a466-4fbfe7268911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:37.007515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32078e98-66db-40b2-a466-4fbfe7268911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:37.008239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32078e98-66db-40b2-a466-4fbfe7268911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:37.009821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32078e98-66db-40b2-a466-4fbfe7268911 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:37.020219 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1452/2888] 10.222.0.13 () {58 vars in 1121 bytes} [Sun Jun 26 03:18:36 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 26 03:18:37.024355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05f3ad-33ce-4fa4-8d1c-6d1fbee2a1c7 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:37.036008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c05f3ad-33ce-4fa4-8d1c-6d1fbee2a1c7 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:37.040784 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a2525-8b69-49d1-8d86-dc04e3e30ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:37.041858 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a2525-8b69-49d1-8d86-dc04e3e30ebf admin admin] REQUEST_METHOD: `PUT` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:37.042248 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a2525-8b69-49d1-8d86-dc04e3e30ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:37.042601 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2a2525-8b69-49d1-8d86-dc04e3e30ebf admin admin] PATH_INFO: `/v3/projects/ab9dc6af1a444a42ab539b5be5fc1922/users/fd58e94ef99d4ce7ba4b860b082bbfe2/roles/0b41a54bd0dd41aaaf09a3a659d1f372` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:37.044703 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a2525-8b69-49d1-8d86-dc04e3e30ebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9dc6af1a444a42ab539b5be5fc1922, user_id=fd58e94ef99d4ce7ba4b860b082bbfe2, role_id=0b41a54bd0dd41aaaf09a3a659d1f372)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:37.047451 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2a2525-8b69-49d1-8d86-dc04e3e30ebf admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:37.064139 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1437/2889] 10.222.0.13 () {60 vars in 1482 bytes} [Sun Jun 26 03:18:37 2022] PUT /identity/v3/projects/ab9dc6af1a444a42ab539b5be5fc1922/users/fd58e94ef99d4ce7ba4b860b082bbfe2/roles/0b41a54bd0dd41aaaf09a3a659d1f372 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:18:41.447675 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c52b62-fbd1-48c9-aadb-fb70115928f1 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:41.448632 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c52b62-fbd1-48c9-aadb-fb70115928f1 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:41.449154 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c52b62-fbd1-48c9-aadb-fb70115928f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:41.491210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-15c52b62-fbd1-48c9-aadb-fb70115928f1 None None] MFA Rules not processed for user `fd58e94ef99d4ce7ba4b860b082bbfe2`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:18:41.494249 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-15c52b62-fbd1-48c9-aadb-fb70115928f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:41.597413 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1453/2890] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:18:41 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:18:41.614823 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a2525-8b69-49d1-8d86-dc04e3e30ebf admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:18:41.616781 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-2b2a2525-8b69-49d1-8d86-dc04e3e30ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:18:41.664743 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:18:41.704304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2a2525-8b69-49d1-8d86-dc04e3e30ebf admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:18:41.711354 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78391609-7de7-42a7-92f2-2367799b962c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:18:41.712135 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78391609-7de7-42a7-92f2-2367799b962c service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:18:41.712304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78391609-7de7-42a7-92f2-2367799b962c service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:18:41.712586 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78391609-7de7-42a7-92f2-2367799b962c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:18:41.774313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78391609-7de7-42a7-92f2-2367799b962c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:18:41.775920 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78391609-7de7-42a7-92f2-2367799b962c service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:18:41.785829 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1438/2891] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:18:41 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:18:51.590437 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:18:51.781185 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=41 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:19:04.217875 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24856c-b268-4354-9600-c7e40abfe526 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:19:04.218454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24856c-b268-4354-9600-c7e40abfe526 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:19:04.218454 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24856c-b268-4354-9600-c7e40abfe526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:19:04.236253 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-7e24856c-b268-4354-9600-c7e40abfe526 None None] MFA Rules not processed for user `6d52b7f924f24f518ad0c9dfb6008260`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:19:04.237444 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7e24856c-b268-4354-9600-c7e40abfe526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:19:04.244981 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1454/2892] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:19:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:19:04.553351 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78391609-7de7-42a7-92f2-2367799b962c service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:19:04.554775 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-78391609-7de7-42a7-92f2-2367799b962c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:19:04.610134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78391609-7de7-42a7-92f2-2367799b962c service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:19:04.613668 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206a60c-5009-4ef3-b22c-4678869a81a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:19:04.614333 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206a60c-5009-4ef3-b22c-4678869a81a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:19:04.614614 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206a60c-5009-4ef3-b22c-4678869a81a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:19:04.614886 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206a60c-5009-4ef3-b22c-4678869a81a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:19:04.616577 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9206a60c-5009-4ef3-b22c-4678869a81a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:19:04.626327 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206a60c-5009-4ef3-b22c-4678869a81a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:19:04.627741 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206a60c-5009-4ef3-b22c-4678869a81a3 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:19:04.635855 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1439/2893] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:19:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:19:14.230892 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:19:14.634894 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=26 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:19:25.941850 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24856c-b268-4354-9600-c7e40abfe526 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:19:25.951690 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24856c-b268-4354-9600-c7e40abfe526 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:19:25.955262 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccec54-62e4-4277-ba70-0db1008c3ffc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:19:25.955899 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccec54-62e4-4277-ba70-0db1008c3ffc service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:19:25.956185 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccec54-62e4-4277-ba70-0db1008c3ffc service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:19:25.956463 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecccec54-62e4-4277-ba70-0db1008c3ffc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:19:25.965796 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccec54-62e4-4277-ba70-0db1008c3ffc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:19:25.967256 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecccec54-62e4-4277-ba70-0db1008c3ffc service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:19:25.977153 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1455/2894] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:19:25 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:19:35.972388 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:20:50.194368 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206a60c-5009-4ef3-b22c-4678869a81a3 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:20:50.202990 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206a60c-5009-4ef3-b22c-4678869a81a3 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:20:50.206925 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdda12-c4f4-4823-a361-2fc7988922a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:20:50.207575 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdda12-c4f4-4823-a361-2fc7988922a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:20:50.207847 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdda12-c4f4-4823-a361-2fc7988922a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:20:50.208120 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bdda12-c4f4-4823-a361-2fc7988922a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:20:50.209879 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-26bdda12-c4f4-4823-a361-2fc7988922a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:20:50.235556 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdda12-c4f4-4823-a361-2fc7988922a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:20:50.237020 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bdda12-c4f4-4823-a361-2fc7988922a4 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:20:50.257542 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1440/2895] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:20:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:20:51.436455 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccec54-62e4-4277-ba70-0db1008c3ffc service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:20:51.444973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecccec54-62e4-4277-ba70-0db1008c3ffc service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:20:51.448544 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3adf6-d2ce-4728-a2de-5721486f1d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:20:51.449169 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3adf6-d2ce-4728-a2de-5721486f1d4c service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:20:51.449447 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3adf6-d2ce-4728-a2de-5721486f1d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:20:51.449717 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3adf6-d2ce-4728-a2de-5721486f1d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:20:51.451468 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-64d3adf6-d2ce-4728-a2de-5721486f1d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:20:51.464322 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3adf6-d2ce-4728-a2de-5721486f1d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:20:51.465712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d3adf6-d2ce-4728-a2de-5721486f1d4c service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:20:51.476205 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1456/2896] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:20:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:21:00.254904 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=15 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:21:01.471529 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:21:03.431722 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdda12-c4f4-4823-a361-2fc7988922a4 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:21:03.440467 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bdda12-c4f4-4823-a361-2fc7988922a4 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:21:03.444261 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff0610-b1cc-49d7-a7cc-3436ed649ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:21:03.444978 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff0610-b1cc-49d7-a7cc-3436ed649ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:21:03.445329 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff0610-b1cc-49d7-a7cc-3436ed649ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:21:03.445672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff0610-b1cc-49d7-a7cc-3436ed649ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:21:03.447571 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-ebff0610-b1cc-49d7-a7cc-3436ed649ed9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:21:03.458180 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff0610-b1cc-49d7-a7cc-3436ed649ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:21:03.459284 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff0610-b1cc-49d7-a7cc-3436ed649ed9 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:21:03.468094 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1441/2897] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:21:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:21:03.552085 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3adf6-d2ce-4728-a2de-5721486f1d4c service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:21:03.561805 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3adf6-d2ce-4728-a2de-5721486f1d4c service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:21:03.565523 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba71470-1f05-4a07-b80c-f96a123acca0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:21:03.566190 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba71470-1f05-4a07-b80c-f96a123acca0 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:21:03.566556 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba71470-1f05-4a07-b80c-f96a123acca0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:21:03.566731 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba71470-1f05-4a07-b80c-f96a123acca0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:21:03.568519 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-dba71470-1f05-4a07-b80c-f96a123acca0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:21:03.582335 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba71470-1f05-4a07-b80c-f96a123acca0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:21:03.583785 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba71470-1f05-4a07-b80c-f96a123acca0 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:21:03.594687 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1457/2898] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:21:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:21:13.464257 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:21:13.589855 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=6 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:21:45.553042 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff0610-b1cc-49d7-a7cc-3436ed649ed9 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:21:45.561841 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff0610-b1cc-49d7-a7cc-3436ed649ed9 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:21:45.565508 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369d5a5-9aca-4114-a372-af0eb2071a24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:21:45.566177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369d5a5-9aca-4114-a372-af0eb2071a24 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:21:45.566468 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369d5a5-9aca-4114-a372-af0eb2071a24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:21:45.567234 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369d5a5-9aca-4114-a372-af0eb2071a24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:21:45.568991 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-0369d5a5-9aca-4114-a372-af0eb2071a24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:21:45.578939 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369d5a5-9aca-4114-a372-af0eb2071a24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:21:45.580343 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369d5a5-9aca-4114-a372-af0eb2071a24 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:21:45.588213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1442/2899] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:21:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:21:46.284547 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d733a-c35d-4743-8585-e81c8dc4846d None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:21:46.285008 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d733a-c35d-4743-8585-e81c8dc4846d None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:21:46.285156 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d733a-c35d-4743-8585-e81c8dc4846d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:21:46.302700 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-e00d733a-c35d-4743-8585-e81c8dc4846d None None] MFA Rules not processed for user `6d52b7f924f24f518ad0c9dfb6008260`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:21:46.303889 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-e00d733a-c35d-4743-8585-e81c8dc4846d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:21:46.311304 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1458/2900] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:21:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:21:55.584671 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:21:56.297600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=3 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:22:42.520030 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369d5a5-9aca-4114-a372-af0eb2071a24 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:22:42.528672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369d5a5-9aca-4114-a372-af0eb2071a24 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:22:42.532819 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391db9d9-317e-49e9-ac9f-16e36d19c55f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:22:42.533449 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391db9d9-317e-49e9-ac9f-16e36d19c55f service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:22:42.533733 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391db9d9-317e-49e9-ac9f-16e36d19c55f service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:22:42.534008 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391db9d9-317e-49e9-ac9f-16e36d19c55f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:22:42.535754 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-391db9d9-317e-49e9-ac9f-16e36d19c55f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:22:42.592605 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391db9d9-317e-49e9-ac9f-16e36d19c55f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:22:42.594014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391db9d9-317e-49e9-ac9f-16e36d19c55f service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:22:42.603657 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1443/2901] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:22:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:22:47.543798 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d733a-c35d-4743-8585-e81c8dc4846d None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:22:47.554096 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d733a-c35d-4743-8585-e81c8dc4846d None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:22:47.557967 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657ba96-b985-48bd-b390-2d98b8184f89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:22:47.558654 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657ba96-b985-48bd-b390-2d98b8184f89 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:22:47.558939 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657ba96-b985-48bd-b390-2d98b8184f89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:22:47.559214 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3657ba96-b985-48bd-b390-2d98b8184f89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:22:47.561627 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3657ba96-b985-48bd-b390-2d98b8184f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:22:47.631210 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657ba96-b985-48bd-b390-2d98b8184f89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:22:47.632848 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3657ba96-b985-48bd-b390-2d98b8184f89 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:22:47.643032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1459/2902] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:22:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:22:50.441793 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391db9d9-317e-49e9-ac9f-16e36d19c55f service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:22:50.443314 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-391db9d9-317e-49e9-ac9f-16e36d19c55f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:22:50.520629 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391db9d9-317e-49e9-ac9f-16e36d19c55f service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:22:50.529007 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a5466-ce66-46f8-a9e4-418cb06511f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:22:50.529677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a5466-ce66-46f8-a9e4-418cb06511f1 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:22:50.529971 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a5466-ce66-46f8-a9e4-418cb06511f1 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:22:50.530297 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a5466-ce66-46f8-a9e4-418cb06511f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:22:50.532352 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-106a5466-ce66-46f8-a9e4-418cb06511f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:22:50.547391 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a5466-ce66-46f8-a9e4-418cb06511f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:22:50.549304 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a5466-ce66-46f8-a9e4-418cb06511f1 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:22:50.561975 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1444/2903] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:22:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:22:53.652666 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657ba96-b985-48bd-b390-2d98b8184f89 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:22:53.654218 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-3657ba96-b985-48bd-b390-2d98b8184f89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:22:53.667195 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3657ba96-b985-48bd-b390-2d98b8184f89 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:22:53.672331 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:22:53.673206 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:22:53.673489 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:22:53.673762 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] PATH_INFO: `/v3/users/faa6dfc6e5c14046b499354e0f07a404` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:22:53.674973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] RBAC: Authorizing `identity:delete_user(user_id=faa6dfc6e5c14046b499354e0f07a404)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:22:53.676884 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:22:53.722885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'faa6dfc6e5c14046b499354e0f07a404'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:53.723537 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'faa6dfc6e5c14046b499354e0f07a404'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:53.730461 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'faa6dfc6e5c14046b499354e0f07a404'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:53.737885 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'faa6dfc6e5c14046b499354e0f07a404'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:53.738530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'faa6dfc6e5c14046b499354e0f07a404'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:53.756313 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1460/2904] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:22:53 2022] DELETE /identity/v3/users/faa6dfc6e5c14046b499354e0f07a404 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:22:53.929009 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a5466-ce66-46f8-a9e4-418cb06511f1 service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:22:53.945557 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a5466-ce66-46f8-a9e4-418cb06511f1 service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:22:54.002390 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:22:54.003174 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:22:54.003458 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:22:54.003734 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] PATH_INFO: `/v3/projects/80a33a31ba6046e78352f5ba9e88b9fe` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:22:54.004890 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a33a31ba6046e78352f5ba9e88b9fe)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:22:54.006807 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:22:54.039256 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:22:54.052811 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a33a31ba6046e78352f5ba9e88b9fe', 'request_id': 'req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.060052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a33a31ba6046e78352f5ba9e88b9fe', 'request_id': 'req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.065640 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a33a31ba6046e78352f5ba9e88b9fe', 'request_id': 'req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.076452 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] The token cache is being invalidate because project 80a33a31ba6046e78352f5ba9e88b9fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:22:54.076746 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.077373 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.078951 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1445/2905] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:22:53 2022] DELETE /identity/v3/projects/80a33a31ba6046e78352f5ba9e88b9fe => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:22:54.083021 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:22:54.084478 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:22:54.097778 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97ac8f6-3bbd-409b-a4c4-e6df57eb634a admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:22:54.152661 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:22:54.154024 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:22:54.154408 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:22:54.154691 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] PATH_INFO: `/v3/users/02c3f36cb4c142a494041163cc76ae79` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:22:54.155862 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] RBAC: Authorizing `identity:delete_user(user_id=02c3f36cb4c142a494041163cc76ae79)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:22:54.157764 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:22:54.162973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:22:54.203525 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02c3f36cb4c142a494041163cc76ae79'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.204239 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02c3f36cb4c142a494041163cc76ae79'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.211032 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02c3f36cb4c142a494041163cc76ae79'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.218730 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02c3f36cb4c142a494041163cc76ae79'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.219338 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02c3f36cb4c142a494041163cc76ae79'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.236753 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1461/2906] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:22:54 2022] DELETE /identity/v3/users/02c3f36cb4c142a494041163cc76ae79 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:22:54.409421 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:22:54.411016 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:22:54.425231 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1c3f7b-52e7-43a7-ad57-b7444a02ed33 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:22:54.482066 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:22:54.482942 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:22:54.483222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:22:54.483498 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] PATH_INFO: `/v3/projects/2fb02fd66e6c4d51be2e1b77085cf32d` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:22:54.484688 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb02fd66e6c4d51be2e1b77085cf32d)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:22:54.486642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:22:54.531587 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb02fd66e6c4d51be2e1b77085cf32d', 'request_id': 'req-bed3a81c-6413-4c2d-8918-b8774d812867'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.538213 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb02fd66e6c4d51be2e1b77085cf32d', 'request_id': 'req-bed3a81c-6413-4c2d-8918-b8774d812867'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.543642 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb02fd66e6c4d51be2e1b77085cf32d', 'request_id': 'req-bed3a81c-6413-4c2d-8918-b8774d812867'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.554264 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] The token cache is being invalidate because project 2fb02fd66e6c4d51be2e1b77085cf32d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80103) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:22:54.554552 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.555175 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:22:54.556677 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1446/2907] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:22:54 2022] DELETE /identity/v3/projects/2fb02fd66e6c4d51be2e1b77085cf32d => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:23:04.233282 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:23:04.551420 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:23:52.784177 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:23:52.785591 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:23:52.854112 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df62c8-7573-4119-b3ae-292c8978795f admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:23:52.911530 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924b0ac-7110-4b6b-8fe5-bda093a39652 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:23:52.912183 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924b0ac-7110-4b6b-8fe5-bda093a39652 service placement] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:23:52.912466 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924b0ac-7110-4b6b-8fe5-bda093a39652 service placement] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:23:52.912743 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924b0ac-7110-4b6b-8fe5-bda093a39652 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:23:52.914531 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-6924b0ac-7110-4b6b-8fe5-bda093a39652 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:23:52.926184 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924b0ac-7110-4b6b-8fe5-bda093a39652 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:23:52.927742 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924b0ac-7110-4b6b-8fe5-bda093a39652 service placement] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:23:52.937501 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1462/2908] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:23:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:23:53.014712 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:23:53.016105 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:23:53.094560 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3a81c-6413-4c2d-8918-b8774d812867 admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:23:53.098643 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e62da-8b2d-42e2-866e-cf6765f91f6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5e65f36f772b4220a909f8fda83ba42e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:23:53.099281 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e62da-8b2d-42e2-866e-cf6765f91f6e service cinder] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:23:53.099561 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e62da-8b2d-42e2-866e-cf6765f91f6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:23:53.099832 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3e62da-8b2d-42e2-866e-cf6765f91f6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:23:53.101573 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6a3e62da-8b2d-42e2-866e-cf6765f91f6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:23:53.159418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e62da-8b2d-42e2-866e-cf6765f91f6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:23:53.160874 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3e62da-8b2d-42e2-866e-cf6765f91f6e service cinder] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:23:53.169757 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1447/2909] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:23:53 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:24:02.933269 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=58 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:24:03.088402 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be16ad3-6bc7-4c9b-9051-6ee21a7288d6 None None] REQUEST_METHOD: `POST` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:24:03.088679 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be16ad3-6bc7-4c9b-9051-6ee21a7288d6 None None] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:24:03.088958 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be16ad3-6bc7-4c9b-9051-6ee21a7288d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:24:03.105837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.auth.core [None req-1be16ad3-6bc7-4c9b-9051-6ee21a7288d6 None None] MFA Rules not processed for user `6d52b7f924f24f518ad0c9dfb6008260`. Rule list: `[]` (Enabled: `True`). {{(pid=80102) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 26 03:24:03.107006 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1be16ad3-6bc7-4c9b-9051-6ee21a7288d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:24:03.113795 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1463/2910] 10.222.0.13 () {58 vars in 957 bytes} [Sun Jun 26 03:24:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 26 03:24:03.165678 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=57 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:24:03.846019 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e62da-8b2d-42e2-866e-cf6765f91f6e service cinder] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:24:03.848176 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6a3e62da-8b2d-42e2-866e-cf6765f91f6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:24:03.863665 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3e62da-8b2d-42e2-866e-cf6765f91f6e service cinder] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:24:03.868831 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337eae57-e893-4422-829f-8629107f6eb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:24:03.869493 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eae57-e893-4422-829f-8629107f6eb2 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:24:03.869777 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eae57-e893-4422-829f-8629107f6eb2 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:24:03.870071 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337eae57-e893-4422-829f-8629107f6eb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:24:03.872081 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-337eae57-e893-4422-829f-8629107f6eb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:24:03.884371 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337eae57-e893-4422-829f-8629107f6eb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:24:03.886200 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337eae57-e893-4422-829f-8629107f6eb2 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:24:03.897177 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1448/2911] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:24:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:24:06.293072 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be16ad3-6bc7-4c9b-9051-6ee21a7288d6 None None] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:24:06.299837 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-1be16ad3-6bc7-4c9b-9051-6ee21a7288d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:24:06.398760 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be16ad3-6bc7-4c9b-9051-6ee21a7288d6 None None] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:24:06.404840 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecc22a-6e82-4352-b6e3-cac4eaef9bff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:24:06.405515 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecc22a-6e82-4352-b6e3-cac4eaef9bff service nova] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:24:06.405806 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecc22a-6e82-4352-b6e3-cac4eaef9bff service nova] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:24:06.406114 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbecc22a-6e82-4352-b6e3-cac4eaef9bff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:24:06.407947 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fbecc22a-6e82-4352-b6e3-cac4eaef9bff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:24:06.421665 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecc22a-6e82-4352-b6e3-cac4eaef9bff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:24:06.423392 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbecc22a-6e82-4352-b6e3-cac4eaef9bff service nova] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:24:06.436472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1464/2912] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:24:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:24:13.892033 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:24:16.431664 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=37 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:24:16.729102 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337eae57-e893-4422-829f-8629107f6eb2 service placement] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:24:16.730656 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-337eae57-e893-4422-829f-8629107f6eb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:24:16.747431 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337eae57-e893-4422-829f-8629107f6eb2 service placement] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:24:16.752788 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4cadc-64b7-4794-8a0c-673aa2458c64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9ee0150dfb3f42709029e42d0fd9105d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:24:16.753960 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4cadc-64b7-4794-8a0c-673aa2458c64 service nova] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:24:16.754283 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4cadc-64b7-4794-8a0c-673aa2458c64 service nova] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:24:16.754558 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4cadc-64b7-4794-8a0c-673aa2458c64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:24:16.756355 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-c3f4cadc-64b7-4794-8a0c-673aa2458c64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:24:16.769489 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4cadc-64b7-4794-8a0c-673aa2458c64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:24:16.771105 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4cadc-64b7-4794-8a0c-673aa2458c64 service nova] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:24:16.782141 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1449/2913] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:24:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:24:26.777223 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:26:25.076102 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecc22a-6e82-4352-b6e3-cac4eaef9bff service nova] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:26:25.077492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-fbecc22a-6e82-4352-b6e3-cac4eaef9bff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:26:25.133948 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbecc22a-6e82-4352-b6e3-cac4eaef9bff service nova] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:26:25.137789 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf3ed-ba1b-4612-b7df-452ad67e2568 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:26:25.138472 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf3ed-ba1b-4612-b7df-452ad67e2568 service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:26:25.138757 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf3ed-ba1b-4612-b7df-452ad67e2568 service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:26:25.139033 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecf3ed-ba1b-4612-b7df-452ad67e2568 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:26:25.140821 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-f8ecf3ed-ba1b-4612-b7df-452ad67e2568 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:26:25.154419 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecf3ed-ba1b-4612-b7df-452ad67e2568 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:26:25.155824 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ecf3ed-ba1b-4612-b7df-452ad67e2568 service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:26:25.166846 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1465/2914] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:26:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:26:26.336345 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4cadc-64b7-4794-8a0c-673aa2458c64 service nova] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:26:26.345726 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4cadc-64b7-4794-8a0c-673aa2458c64 service nova] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:26:26.349273 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc99248-a4d6-49c0-ba81-7eb7ed5673e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:26:26.349901 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc99248-a4d6-49c0-ba81-7eb7ed5673e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:26:26.350222 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc99248-a4d6-49c0-ba81-7eb7ed5673e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:26:26.350501 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc99248-a4d6-49c0-ba81-7eb7ed5673e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:26:26.352233 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7fc99248-a4d6-49c0-ba81-7eb7ed5673e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:26:26.365212 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc99248-a4d6-49c0-ba81-7eb7ed5673e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:26:26.366664 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc99248-a4d6-49c0-ba81-7eb7ed5673e7 service neutron] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:26:26.377672 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1450/2915] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:26:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:26:35.162450 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=26 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:26:36.372574 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=6 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:26:37.371940 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf3ed-ba1b-4612-b7df-452ad67e2568 service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:26:37.381230 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ecf3ed-ba1b-4612-b7df-452ad67e2568 service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:26:37.384813 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfe250-34fe-4d91-9c5b-a415e68ecbad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8ff708d6758d4f7b913e763824f3b791', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:26:37.385441 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe250-34fe-4d91-9c5b-a415e68ecbad service neutron] REQUEST_METHOD: `GET` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:26:37.385723 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe250-34fe-4d91-9c5b-a415e68ecbad service neutron] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:26:37.386001 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cfe250-34fe-4d91-9c5b-a415e68ecbad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:26:37.388076 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-c0cfe250-34fe-4d91-9c5b-a415e68ecbad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:26:37.452197 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfe250-34fe-4d91-9c5b-a415e68ecbad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:26:37.453655 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cfe250-34fe-4d91-9c5b-a415e68ecbad service neutron] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:26:37.463023 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1466/2916] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 26 03:26:43.609492 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc99248-a4d6-49c0-ba81-7eb7ed5673e7 service neutron] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:26:43.611419 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-7fc99248-a4d6-49c0-ba81-7eb7ed5673e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:26:43.629699 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc99248-a4d6-49c0-ba81-7eb7ed5673e7 service neutron] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:26:43.635052 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:26:43.636027 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:26:43.636387 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:26:43.636738 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] PATH_INFO: `/v3/users/6d52b7f924f24f518ad0c9dfb6008260` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:26:43.638332 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d52b7f924f24f518ad0c9dfb6008260)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:26:43.640338 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:26:43.687418 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d52b7f924f24f518ad0c9dfb6008260'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:43.688346 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d52b7f924f24f518ad0c9dfb6008260'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:43.688970 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d52b7f924f24f518ad0c9dfb6008260'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:43.700519 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d52b7f924f24f518ad0c9dfb6008260'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:43.707902 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d52b7f924f24f518ad0c9dfb6008260'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:43.720082 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1451/2917] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:26:43 2022] DELETE /identity/v3/users/6d52b7f924f24f518ad0c9dfb6008260 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:26:43.899268 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfe250-34fe-4d91-9c5b-a415e68ecbad service neutron] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:26:43.908831 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cfe250-34fe-4d91-9c5b-a415e68ecbad service neutron] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:26:43.967082 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:26:43.967973 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:26:43.968260 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:26:43.968540 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] PATH_INFO: `/v3/projects/4a6b41994f444cafb21d3d6e395b6893` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:26:43.969709 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a6b41994f444cafb21d3d6e395b6893)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:26:43.971658 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:26:44.017058 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a6b41994f444cafb21d3d6e395b6893', 'request_id': 'req-7cb9d2b1-b44a-4b15-88a6-880662170ed8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.022989 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a6b41994f444cafb21d3d6e395b6893', 'request_id': 'req-7cb9d2b1-b44a-4b15-88a6-880662170ed8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.030681 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a6b41994f444cafb21d3d6e395b6893', 'request_id': 'req-7cb9d2b1-b44a-4b15-88a6-880662170ed8'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.041969 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] The token cache is being invalidate because project 4a6b41994f444cafb21d3d6e395b6893 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:26:44.042287 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.042915 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.044492 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1467/2918] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:26:43 2022] DELETE /identity/v3/projects/4a6b41994f444cafb21d3d6e395b6893 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:26:44.048422 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:26:44.049824 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:26:44.062830 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5827818d-1e2c-446a-a0ba-fd373856a66f admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:26:44.117966 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:26:44.118865 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:26:44.119155 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:26:44.119434 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] PATH_INFO: `/v3/users/fd58e94ef99d4ce7ba4b860b082bbfe2` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:26:44.120618 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd58e94ef99d4ce7ba4b860b082bbfe2)` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:26:44.122608 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:26:44.168014 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd58e94ef99d4ce7ba4b860b082bbfe2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.168719 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd58e94ef99d4ce7ba4b860b082bbfe2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.169359 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd58e94ef99d4ce7ba4b860b082bbfe2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.181041 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd58e94ef99d4ce7ba4b860b082bbfe2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.188077 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.notifications [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd58e94ef99d4ce7ba4b860b082bbfe2'} {{(pid=80103) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.190276 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:26:44.203407 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1452/2919] 10.222.0.13 () {58 vars in 1223 bytes} [Sun Jun 26 03:26:44 2022] DELETE /identity/v3/users/fd58e94ef99d4ce7ba4b860b082bbfe2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:26:44.376366 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] Authenticating user token {{(pid=80102) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:26:44.377799 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.fernet_utils [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80102) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:26:44.391124 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9d2b1-b44a-4b15-88a6-880662170ed8 admin admin] Validating token access rules against request {{(pid=80102) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:26:44.412292 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:26:44.449417 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a05c37bdf694480a8a14de0b18b2da4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'becb24f8723f432e9a3c4c740a191a0b', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80102) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:26:44.450305 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:26:44.450588 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:26:44.450861 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] PATH_INFO: `/v3/projects/ab9dc6af1a444a42ab539b5be5fc1922` {{(pid=80102) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:26:44.452019 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab9dc6af1a444a42ab539b5be5fc1922)` {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:26:44.453896 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] RBAC: Authorization granted {{(pid=80102) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:26:44.498271 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab9dc6af1a444a42ab539b5be5fc1922', 'request_id': 'req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.503716 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab9dc6af1a444a42ab539b5be5fc1922', 'request_id': 'req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.512087 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab9dc6af1a444a42ab539b5be5fc1922', 'request_id': 'req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a'} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.533712 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] The token cache is being invalidate because project ab9dc6af1a444a42ab539b5be5fc1922 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80102) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 26 03:26:44.534029 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.534670 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG keystone.notifications [None req-b0a9af34-5e42-43d4-8bf9-54b53d08db8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80102) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 26 03:26:44.536231 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: [pid: 80102|app: 0|req: 1468/2920] 10.222.0.13 () {58 vars in 1232 bytes} [Sun Jun 26 03:26:44 2022] DELETE /identity/v3/projects/ab9dc6af1a444a42ab539b5be5fc1922 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 26 03:26:54.202581 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=4 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:26:54.530600 dv-cinder-846842-4-iscsi devstack@keystone.service[80102]: DEBUG dbcounter [-] [80102] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80102) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 26 03:28:57.805992 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] Authenticating user token {{(pid=80103) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 26 03:28:57.807459 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:28:57.944251 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b32bdf4-9cbd-463e-adbd-8d9c7342fd7d admin admin] Validating token access rules against request {{(pid=80103) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 26 03:28:58.001313 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a29d5c7-c25d-4f24-b665-5a9f89ab9958 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19ce6f259c22441486bfc04811a271b8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '33208982d2b04888aa7d4e1ca0d5df53', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80103) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 26 03:28:58.001969 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29d5c7-c25d-4f24-b665-5a9f89ab9958 service placement] REQUEST_METHOD: `GET` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 26 03:28:58.002279 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29d5c7-c25d-4f24-b665-5a9f89ab9958 service placement] SCRIPT_NAME: `/identity` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 26 03:28:58.002559 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a29d5c7-c25d-4f24-b665-5a9f89ab9958 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80103) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 26 03:28:58.004319 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.fernet_utils [None req-6a29d5c7-c25d-4f24-b665-5a9f89ab9958 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80103) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 26 03:28:58.015769 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29d5c7-c25d-4f24-b665-5a9f89ab9958 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 26 03:28:58.017340 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a29d5c7-c25d-4f24-b665-5a9f89ab9958 service placement] RBAC: Authorization granted {{(pid=80103) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 26 03:28:58.027134 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: [pid: 80103|app: 0|req: 1453/2921] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Jun 26 03:28:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 26 03:29:08.022938 dv-cinder-846842-4-iscsi devstack@keystone.service[80103]: DEBUG dbcounter [-] [80103] Writing DB stats keystone:SELECT=89 {{(pid=80103) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}